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1  Introduction 


Background 

Many  Army  personnel  can  be  classified  as  knowledge  workers — people  who  produce 
not  tangible  products,  but  some  form  of  processed  or  enhanced  information,  often  using 
processes  that  allow  a  high  degree  of  individual  discretion  in  task  performance. 
Knowledge  workers  make  decisions  that  significantly  impact  organizational  resources 
and  are  themselves  a  significant  and  costly  resource;  knowledge  workers  compose  43 
percent  of  the  white-collar  sector,  which  in  turn  comprises  67  percent  of  the  service 
sector  (Roach  1991). 

Knowledge  work  is  the  area  that  offers  the  greatest  opportunity  to  increase 
productivity  within  the  U.S.  workforce  (Drucker  1974).  Ongoing  research  at  the  U.S. 
Army  Construction  Engineering  Research  Laboratories  (USACERL)  is  developing  an 
integrated  software  program  designed  to  improve  the  performance  of  Army  knowledge 
workers.  The  Knowledge  Worker  System  (KWS)  is  a  computer-based  performance 
support  environment  (PSE)  designed  to  document,  coordinate,  and  execute  the 
business  processes  assigned  to  workgroups.  KWS  is  an  integrated  set  of  automation 
systems  that  guides  Army  action  officers  through  the  course  of  their  daily  tasks  by 
helping  them  organize,  prioritize,  and  execute  their  work  efficiently  and  effectively. 
KWS  can  help  meet  the  need  for  a  wide  range  of  functional  process  improvements  and 
training  requirements  by  providing  institutional  knowledge  on  an  as-needed,  individ¬ 
ualized  basis. 

While  KWS  promises  to  offer  significant  benefits,  installing  any  new  system  involves 
a  commitment  of  both  time  and  resources  that  must  justify  itself  economically; 
i.e.,  benefits  must  outweigh  costs.  Part  of  the  KWS  research  effort  is  to  identify 
appropriate  metrics  that  indicate  the  impact  of  the  system  on  workgroups.  This  study 
undertook  to  investigate  techniques  to  measure  whether  the  implementation  and/or 
continued  use  of  a  PSE  for  knowledge  workers  such  as  KWS  is  justified. 
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Objectives 

The  objective  of  this  research  is  to  identify  appropriate  methods  to  evaluate  the 
feasibility  of  implementing,  or  continuing  the  use  of,  a  performance  support  environ¬ 
ment  for  knowledge  workers. 


Approach 

A  broad  review  of  literature  in  productivity  measurement  was  conducted,  including 
areas  such  as  white  collar  productivity,  organizational  psychology,  information 
economics,  industrial  engineering,  economics  of  technology,  quality  management,  and 
decision  theory.  Several  techniques  for  productivity  measurement  were  evaluated  for 
their  applicability  to  the  environments  in  which  Army  knowledge  workers  operate. 

Mode  of  Technology  Transfer 

The  findings  from  this  study  will  be  incorporated  into  future  USACERL  work  units 
that  address  the  development  of  the  Knowledge  Worker  System. 
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2  Issues  in  Evaluation 


“What  gets  measured  gets  attention,  particularly  when  rewards  are  tied  to  measures.” 

(Eccles,  in  Helton  1992) 


Why  Measure 

“Productivity”  is  a  fairly  intuitive  notion  that  can  be  defined  as  the  relationship  of  the 
outputs  to  the  inputs  used  in  production  (National  Research  Council  1994).  If  outputs 
and  inputs  are  well  quantified,  that  relationship  might  simply  be  defined — and 
compared — mathematically,  as  output  divided  by  input  (Sink  1984). 

The  goal  of  productivity  measurement  is  to  determine  whether  organizations  can 
obtain  the  same  output  with  fewer  resources,  or  increase  output  while  holding  resource 
levels  constant  (Lau  1988).  With  this  goal  in  mind,  three  reasons  for  evaluating 
productivity  are: 

1.  To  identify  potential  improvements 

2.  To  determine  how  to  allocate  resources 

3.  To  determine  how  well  organizational  goals  are  being  met  (Sardina  and  Vrat 
1987). 

Within  the  Federal  government,  the  Bureau  of  Labor  Statistics  monitors  productivity 
by  collecting  output  measures,  identifying  resource  requirements,  and  determining 
estimated  production  goals  (Forte  1992).  Objective  measures  of  output,  resource 
levels,  and  production  goals  are  benchmarks  that  form  a  baseline  to  compare  and 
improve  productivity  levels.  Without  such  benchmarks,  organizations  could  not  set 
goals  or  allocate  resources  since,  “without  productivity  objectives,  a  business  does  not 
have  direction.  Without  productivity  measurement,  it  does  not  have  control”  (Drucker 
1974).  It  is  this  baseline  of  knowledge  worker  productivity  that  must  be  determined 
(Bridges  1992). 
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Problems  in  Evaluating  Knowledge  Work 

Measurement  and  evaluation  of  knowledge  work  is  a  difficult  problem  (Drucker  1974; 
Magliola-Zoch  1984;  Rittenhouse  1992;  Sassone  1991;  Sink  1985;  Thomas  and  Baron 
1994).  Many  of  the  problems  relate  to  the  very  nature  of  knowledge  work:  its  inputs 
are  not  clearly  definable;  it  generates  intangible  outputs;  and  it  allows  a  high  degree 
of  discretion  on  the  part  of  the  performer  (Beruvides  and  Sumath  1987).  Knowledge 
work  is  often  complex  and  nonroutine,  and  commonly  requires  the  contribution  of 
several  people  to  complete  a  given  task.  All  these  characteristics  make  norms  and 
standards  difficult  to  establish,  and  performance  hard  to  measure. 

The  net  results  of  this  complexity  is  disagreement  about  what  to  evaluate.  Frequently, 
inappropriate  metrics  are  applied  to  knowledge  work  simply  because  the  particular 
indicator  in  use  is  easy  to  quantify  (Rittenhouse  1992).  The  essential  component  of 
quality  is  often  ignored  due  to  the  difficulty  in  measuring  this  attribute. 

In  1994,  the  National  Research  Council  commissioned  a  committee  to  investigate  the 
apparent  meager  increases  in  productivity  in  the  service  sector,  despite  the  large 
amounts  of  money  spent  on  information  technology  for  this  group.  The  committee 
concluded  that,  to  a  large  degree,  these  disappointing  productivity  results  arise  from 
inappropriate  metrics: 

Productivity  data  do  not  capture  important  elements  of  service  output.  Key  among 
these  are  the  capacities  to  handle  increased  complexity  and  to  provide  improved 
timeliness,  flexibility,  response  times,  reliability,  or  safety  for  employees, 
customers,  or  the  general  public. 

If  these  and  other  factors  are  accounted  for,  the  resulting  figures  might  demonstrate 
a  strong  increase  in  service-sector  performance  through  information  technology 
investments  (Peterson  1994,  p  7). 


Individual  vs.  Group  Measures 

Rittenhouse  (1992)  and  Sassone  (1991)  identify  the  workgroup  as  the  appropriate  level 
at  which  to  measure  the  performance  of  knowledge  workers.  Rittenhouse  observes 
that  individual  measurements  are  not  particularly  useful  since  increases  in  an 
individual’s  productivity  does  not  necessarily  transfer  upward  within  an  organization. 
For  this  reason,  this  study  concentrated  on  group  evaluation. 
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Several  practicioners  recommend  the  use  of  a  family  of  measures.  The  director  of 
quality  measurement  and  improvement  at  USAA  has  credited  the  ability  to  track  both 
individual  performance  (in  some  categories)  and  group  performance  (in  others)  as  a 
factor  in  the  organization’s  finalist  position  in  the  Malcolm  Baldridge  National  Quality 
Award  for  multiple  years  (Helton  1992). 

Helton  (1992)  recommends  an  approach  that  includes  the  following  steps:  (1)  select 
the  group  involved  in  the  work  to  be  tracked,  (2)  help  this  group  select  several 
measures  appropriate  to  the  work,  (3)  help  the  group  clearly  define  the  measures, 
frequency  of  measurement,  and  whether  benchmarking  is  appropriate,  and, 
(4)  document  the  results. 
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3  Evaluation  Strategy 

Suggested  Approach 

In  earlier  work,  Thomas  and  Baron  (1994)  concluded  that  knowledge  work  perfor¬ 
mance  measurement  required  special  evaluation  techniques.  No  single  technique  is 
appropriate  for  all  types  of  knowledge  work.  The  authors  suggested  that  the  first  step 
in  performance  measurement  within  knowledge  work  environments  was  work 
categorization.  Thomas  and  Baron  identified  eight  components  of  knowledge  work 
relevant  to  categorizing  work  groups: 

1.  Decisionmaking 

2.  Complexity 

3.  Knowledge  use 

4.  Structure 

5.  Repetition 

6.  Volume 

7.  Time  per  job 

8.  Skill  level. 

This  earlier  work  suggests  that  a  “toolkit”  of  evaluation  techniques  is  appropriate  to 
evaluate  the  feasibility  of  a  performance  support  environment  for  knowledge  workers. 
Industrial  engineering,  industrial  management,  and  operations  research  have  yielded 
a  variety  of  evaluation  techniques,  each  applicable  to  specific  work  group  settings. 

Before  determining  which  techniques  fit  a  particular  work  group,  the  first  step  should 
be  to  categorize  the  type  of  knowledge  work  performed  within  that  environment. 
Reducing  the  eight  work  components  to  four  key  components  reduces  the  categoriza¬ 
tion  effort,  while  still  maintaining  clear  and  useful  categories: 

1.  Complexity  incorporates  the  first  three  of  the  original  eight  components 
(decisionmaking,  complexity,  and  knowledge  use).  Issues  relevant  to  evaluating 
the  degree  of  complexity  in  a  process  include:  how  much  knowledge  is  applied  in 
this  process;  how  much  thought  must  be  applied  to  perform  the  job;  and  how 
much  information  is  required  to  complete  the  process. 
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2.  Volume  reflects  the  number  of  times  a  targeted  activity  occurs  within  a  given 
time  frame.  Issues  relevant  to  evaluating  the  volume  level  are:  whether  the 
activity  is  performed  frequently  within  a  month,  a  week,  a  quarter,  or  a  1-year 
span. 

3.  Time  per  job  is  the  time  spent  completing  the  job,  start  to  finish.  When 
evaluating  this  component,  the  issue  is  whether  the  relative  time  spent  on  each 
task  is  high,  medium,  or  low. 

4.  Repetitions  indicate  the  frequency  with  which  a  process  is  performed.  Evaluating 
this  attribute  involves  how  often  the  same  process  is  repeated  (i.e,  performed 
without  variation  from  previous  occurrences). 

The  remaining  two  components,  Structure  and  Skill  Level ,  can  be  eliminated  from 
work  categorization  exercise  without  sacrificing  meaningful  results  since  structure 
appears  to  vary  significantly  within  knowledge  work  environments  and  skill  level  is 
more  relevant  to  blue-collar  work  environments  than  to  knowledge  work. 

Helton  (1992)  suggests  four  criteria  for  the  categorization  of  knowledge  work: 

1.  Work  Range  means  the  scope  of  the  work  performed.  Questions  to  ask  when 
evaluating  the  work  are:  Are  tasks  repetitive?  Are  tasks  cyclic? 

2.  Work  Structure  indicates  whether  work  objectives  are  shifting  or  fixed.  A 
question  to  ask:  How  changeable  are  work  goals? 

3.  Control  reflects  the  degree  of  discretion  allowed.  Some  questions  to  ask  include: 
How  planned  is  the  work?  How  much  freedom  of  choice  is  allowed  in  how,  when, 
or  where  the  work  is  done? 

4.  Cognitive  Effort  indicates  the  degree  of  reasoning  difficulty  used  to  solve  work 
problems.  Cognitive  effort  can  be  ranked  as:  very  substantial,  substantial, 
intermediate,  or  limited. 

Helton  uses  these  criteria  to  classify  work  into  one  of  four  types  of  white-collar  work: 
Specialist,  Professional,  Support  Staff,  and  Clerical  Staff  (Table  1).  Helton’s  categories 
correspond  roughly  with  the  suggested  framework:  Work  Range  correlates  with  Time 
per  Job;  (2)  Work  Structure  correlates  with  Volume  and  Repetitions;  (3)  Controls 
correspond  with  Structure,  and  (4)  Cognitive  Effort  correlates  with  Complexity. 
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Table  1 .  Principle  attributes  by  white-collar  work  type. 


Work  Range 

Specialist* 

Professional 

Support 

Clerical  | 

Non-repetitive 

P 

P 

S 

— 

Repetitive 

. 

S 

P 

Non-routine 

P 

P 

s 

111*!! 

Routine 

■■■II 

Individual 

P 

p 

Group 

HHHH 

_ s _ 

E 

_ £ _ 

Sequence-Dependent 

iiiiiwaa 

E 

Work  Structure 

Shifting  Objectives 

BOB 

liKasiaii 

Fixed  Objectives 

IlliBli 

S 

pfpojp 

P 

Control 

W§WM§ 

Discretionary 

p 

P 

S 

S 

Non-discretionary 

— — 1 
■llllIllllM 

P 

m 

Cognitive  Effort 

1— gj 

Very  Substantial 

HOI 

s 

H  fH 

lilBlili 

Substantial 

p 

■■■I 

11811® 

Limited 

■— 

i— 

_ £ _ 

*P  =  primary;  S  =  secondary;  E  =  ( 

jlther  1 

Selection  of  Evaluation  Techniques 

Table  2  and  Figure  1  show  how  work  is  categorized  by  four  work  components  to 
identify  three  resulting  types  of  work  environments:  Knowledge  Work,  Production 
Office,  and  Proceduralized  Work  (formerly  called  blue  collar  work).  Each  of  these 
environments  exhibits  a  range  of  values  for  each  of  the  four  work  components. 
Proceduralized  (blue  collar)  Work  is  included  only  as  a  point  of  comparison  with  the 
two  knowledge  work  environments. 

Once  a  work  group  has  been  categorized  by  the  four  composite  knowledge  work 
attributes,  the  next  step  is  to  choose  an  appropriate  evaluation  tool.  The  workgroup 
itself  must  participate  in  establishing  the  particular  metrics  to  be  used  for  evaluation. 
Besides  being  the  best  judges  of  appropriate  metrics,  the  more  involved  the  group  is, 
the  less  its  members  are  likely  to  feel  threatened  by  the  study  (Anthony  1984;  Bernard 
1986). 
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Table  2.  Work  type  categorized  by  component  and  degree. 


Complexity 

Time  per  Job 

Re 

setitions 

Volume  per  Worker  | 

Component  Level  -> 

H 

M 

n 

H 

M 

n 

H 

M 

L 

H 

M 

D 

Knowledge  Work 

X 

X 

X 

D 

X 

X 

D 

X 

Production  Office 

D 

D 

■ 

X 

X 

X 

D 

X 

X 

Proceduralized  Work 

X 

X 

X 

n 

X 

ID 

X 

X 

Information  Economics 

Economists  Parker, 

Benson,  and  Trainor 
(1988)  proposed  Infor¬ 
mation  Economics  (IE) 
as  a  framework  that 
quantifies  and  weights 
intangible  benefits  and 
costs  of  information 
technology  (IT)  alter¬ 
natives.  IE  is  intended 
as  an  approach  for 
evaluating  an  organiza¬ 
tion’s  IT  investments. 

Standard  cost-benefit  analysis,  the  conventional  tool  for  evaluating  IT  projects,  is 
based  on  accounting-type  data  such  as  projected  costs  and  benefits,  and  estimated 
return  on  investment  (ROI).  However,  this  standard  approach  ignores  a  number  of 
factors  that  may  determine  the  success  or  failure  of  a  project  simply  because  these 
factors  do  not  lend  themselves  to  dollar  quantification.  Traditional  cost-benefit 
analysis  is  not  adequate  for  evaluation  of  innovative  systems  such  as  performance 
support  environments  (Parker,  Benson,  and  Trainor  1988). 

Information  Economics  can  be  defined  as  a  collection  of  computational  tools  that  allow 
rational  comparison  of  benefits  and  costs  of  IT  projects.  Parker,  Benson,  and  Trainor 
claim  that  IE  goes  beyond  cost-benefit  analysis  by  providing  a  decisionmaking 
structure  that  separates  information  technology  justification  from  technology 
feasibility  for  a  particular  project.  In  other  words,  IE  allows  an  organization  to  make 
two  important  distinctions:  (1)  what  this  project  is  worth  to  the  organization,  and  (2) 
whether  the  organization  has  the  resources  necessary  to  complete  the  project.  In 


Figure  1 .  Work  categorized  by  four  work  components. 
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terms  of  justifying  a  PSE,  Information  Economics  is  valuable  mainly  because  of  the 
tools  it  provides  for  determining  an  answer  to  the  first  question. 

Figure  2  shows  how  IE  focuses  on  value  rather  than  the  more  limited  concept  of  benefit 
and  hard  dollar  savings.  The  model  expands  the  traditional  cost-benefit  analysis  to 
include  quantification  of  intangible  benefits  and  risks  of  both  business  and  technical 
issues.  Dollar  savings  are  important,  but  intangible  elements  such  as  increased 
knowledge  worker  productivity,  improved  communications,  and  enhanced  quality 
should  be  evaluated  by  organizations  as  they  make  IT  decisions.  IE  typically  suggests 
six  classes  of  value: 

1.  Enhanced  ROI  is  similar  to  standard  return  on  investment  but  expanded  to 
include  additional  methods  (value  acceleration,  value  linking,  and  value 
restructuring). 

2.  Strategic  Match  is  a  measure  of  how  closely  aligned  the  project  is  to  the 
organization’s  strategic  goals. 

3.  Competitive  Advantage  estimates  the  degree  to  which  the  project  provides  an 
advantage  in  the  marketplace,  and  can  be  viewed  as  an  improvement  in  the 
product  or  service  of  the  organization,  or  as  a  sharpening  of  the  focus  of  its  vision. 

4.  Management  Information  reflects  the  value  of  the  information,  or  the  improved 
information  that  the  project  is  expected  to  provide.  The  more  essential  to  the 
functioning  of  the  organization  the  project’s  information  is,  the  higher  its  value. 


Services  to  Create  Value 
Recovery  for  Services 


Tangibles 
Intangible  Benefits 
+  Intangible  Risks 


Net  value  produced 

Business  feasibility . 
&  economic  analysis 

Business  Domain 


Intangible  Benefits 
+  Intangible  Risks 

Net  value  produced 

Technical  feasibility 
&  economic  viability 

Technical  Domain 


-  adapted  from  Parker  &  Benson,  1988 


Figure  2.  Information  economics. 


USACERL  TR  95/32 


17 


5.  Competitive  Response  is  an  estimate  of  the  consequences  of  not  implementing  the 
project.  (This  is  another  class  of  value  that  has  direct  reference  to  nongovern¬ 
mental  organizations  in  that  it  relates  to  the  market  share  and  related  issues. 
However,  there  can  be  many  consequences  of  not  implementing  projects  in  the 
government.  This  can  be  viewed  as  the  nonmarketing  consequences  of  inactivity. 
An  organization  that  appears  unable  to  handle  its  work  effectively  may  no  longer 
have  that  work  to  do  or  it  may  find  itself  reorganized  into  some  other  organiza¬ 
tion.) 

6.  Strategic  IS  Architecture  assumes  that  there  is  some  strategic  plan  for  the 
information  systems  of  the  organization,  and  is  used  to  measure  how  the  project 
fits  into  and/or  complements  that  overall  plan. 

Each  of  these  value  classes  is  assigned  a  weight  for  the  organization  in  which  the  IE 
analysis  is  occurring.  Eventually,  every  proposed  project  will  be  evaluated  in  each 
value  class  for  its  effect  and  then  a  summarized  score  is  calculated  for  each  project. 

In  applying  IE  concepts  in  a  cost-benefit  analysis,  an  organization  typically  begins 
from  its  business  vision  and  then  establishes  criteria  and  weights  that  can  be  used  in 
deciding  which  IT  investments  best  fit  the  vision.  When  applying  IE  to  the  justifica¬ 
tion  of  a  PSE,  the  starting  point  is  a  set  of  potential  benefits  that  can  be  attributed  to 
implementation  of  the  PSE.  Consider  the  following  example: 

PERFECT  Company  places  great  value  on  never  making  a  mistake  on  a  customer’s 
order.  Their  vision  is  focused  on  this  error-free  goal.  Their  evaluation  of  any 
project  will  weigh  heavily  its  effect  on  perfection.  CHEAP  Company  on  the  other 
hand  takes  great  pride  in  offering  the  cheapest  price  on  the  market.  Consequently, 
this  company  will  weigh  any  project  heavily  for  its  effect  on  price. 

This  example  shows  that  what  the  user  deems  important  must  be  established  as  a  first 
step  in  evaluation.  Once  an  evaluation  team  has  determined  what  the  user  finds 
important  in  a  potential  IT  project,  the  selection  of  appropriate  tools  and  measures  will 
follow.  In  the  example,  the  overriding  concern  of  the  PERFECT  Company  is 
perfection.  Logically,  the  business  will  look  for  measures  that  reflect  the  level  of 
perfection. 

Suppose  IE  is  applied  to  the  justification  of  KWS.  One  of  the  benefits  of  KWS  is  an 
improvement  in  scheduling  efficiency.  If  one  member  of  the  evaluation  team  rates  this 
benefit  as  very  important,  then  one  of  the  metrics  for  determining  KWS  success  for 
this  rater  will  be  the  improvement  of  scheduling.  A  point  central  to  the  IE  concept  is 
that  justification  of  IT  is  based  on  factors  important  to  the  IT  user. 
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There  are  four  steps  in  performing  the  IE-derived  cost-benefit  process: 

1.  Identify  tangible  and  intangible  benefits/risks. 

2.  Assign  relative  weights  to  each  identified  benefit  and  risk. 

3.  Rate  each  benefit  and  risk  in  terms  of  importance  to  the  organization. 

4.  Total  the  individual  scores  and  identify  which  alternative  has  the  highest  sum. 
Document  the  rationale  behind  each  ranking  and  weight  to  create  an  audit  trail 
of  the  decisionmaking  process. 

The  application  of  IE  cost-benefit  analysis  to  the  evaluation  of  a  performance  support 
environment  follows  the  same  four-step  procedure.  The  group  that  performs  this 
analysis  should  be  comprised  of  representatives  of  top  management  and  personnel  who 
will  primarily  use  the  PSE. 

The  value  classes  supported  by  the  PSE  must  be  identified.  The  following  classes  are 
proposed: 

•  Enhanced  ROl  includes  any  of  the  following  items: 

cost  avoidance 

reductions  in  operating  costs. 

•  Performance  Improvements  addresses  any  of  the  following  items: 

improved  timeliness 

-  improved  quality 
improved  decisionmaking. 

•  Strategic  Match  is  a  measure  of  how  closely  the  PSE  aligns  with  the  organiza¬ 
tion’s  strategic  goals. 

•  Strategic  IS  Architecture  is  a  measure  of  how  the  proposed  PSE  fits  into  and/or 
complements  the  organization’s  overall  strategic  plan  for  the  information 
systems.  In  the  absence  of  a  plan,  this  factor  may  be  evaluated  in  accordance 
with: 

long-term  support  requirements 

-  disruption  to  business  during  start-up  period 

-  on-going  training  requirements. 

The  major  benefits  of  the  PSE  must  be  clearly  identified.  The  key  KWS  benefits  are: 

1.  Improved  Effectiveness  increases  intellectual  specialization  within  the  organiza¬ 
tion,  and  is  sometimes  referred  to  as  “doing  the  right  thing.” 
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2.  Reduced  Rework  decreases  the  amount  of  work  that  must  be  redone  due  to  error. 

3.  Improved  Efficiency  means  that  the  same  work  can  be  accomplished  in  less  time, 
and  is  sometimes  referred  to  as  “doing  the  thing  right.” 

4.  Improved  Focus  makes  more  time  available  to  devote  to  the  primary  mission  or 
function  of  the  workgroup,  rather  than  to  tangential  activities. 

5.  Work  Elimination  allows  KWS  either  to  eliminates  the  need  for  some  tasks  or  to 
accomplish  them  automatically. 

The  worksheet  shown  in  Figure  3  reflects  these  proposed  value  classes  and  the  KWS 
benefits.  The  key  point  for  the  justification  of  a  PSE  using  the  IE  concept  of  cost- 
benefit  analysis  is  to  perform  this  type  of  analysis  at  the  beginning  of  the  project.  Use 
the  results  of  the  IE  analysis  to:  (1)  gain  consensus  among  management  and  primary 
users  about  the  expected  benefits,  (2)  influence  the  choice  of  performance  metrics  and, 
(3)  create  an  audit  trail  of  the  information  and  decisions  that  initiated  the  PSE. 

Appendix  A  provides  a  taxonomy  of  KWS  benefits,  capabilities,  and  functions  as  well 
as  an  explanation  of  the  worksheet  example  used  here.  Appendix  B  gives  more 
information  on  Information  Economics. 


20 


USACERL  TR  95/32 


Value 


I  importance  of  the  value 
to  the  organization 


EXPLANATIONS  OF  CODES 


Value  Codes 


Relation  Codes 


0  Not  Relevant 

1  Not  Important 

2  Little  Importance 

3  Moderate  Importance 

4  Important 

5  Very  Important 


0  No  Relation 

1  Minimal  Relation 

2  Some  Relation  . 

3  Moderate  Relation 

4  Strong  Relation 

5  Absolute  Relation 


Figure  3.  Scoresheet  for  KWS  implementation. 


SCORE  FOR  KWS  IMPLEMENTATION 


VALUE  CLASSES 


ENHANCED  RO 


PERFORMANCE  IMPROVEMENTS 


STRATEGIC  MATCH 


STRATEGIC  IS  ARCHITECTURE 


KWS  BENEFITS 


Relation  to 
numbered  value 


Benefit 


Ltr.  Name 


A  Improved  Effectiveness 


B  Reduced  Rework 


C  Improved  Efficiencv 


D  Improved  Focus 


E  Work  Elimination 
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4  Toolkit  of  Evaluation  Techniques 


Several  techniques  should  be  included  in  a  toolkit  for  evaluating  the  feasibility  of 
implementing  a  PSE  for  knowledge  workers.  Each  technique  is  described  here. 


Work  Profile  Analysis 

Work  Profile  Analysis  is  part  of  the  hedonic  wage  model,  as  applied  to  the  justification 
of  an  office  information  system  (OIS)  by  Peter  Sassone  (1987,  1992,  1992).  This 
application  of  the  hedonic  wage  model  assumes  that  an  OIS  can  both  decrease  the 
amount  of  time  required  to  complete  a  given  task  and  facilitate  restructuring  of  work 
assignments.  Both  of  these  changes  are  postulated  to  result  in  higher  efficiency. 
Professionals  have  more  time  to  perform  work  in  their  specific  specialties  and  spend 
less  time  on  routine  and/or  nonproductive  tasks.  The  combination  of  the  OIS  and 
restructuring  within  an  office  can  correct  the  misallocation  of  time  spent  by  white- 
collar  professionals  on  lower-value  activities. 

The  premise  of  the  hedonic  wage  model,  as  applied  to  OIS,  is  that  the  value  of  an 
information  system  originates  in  the  value  of  the  activities  performed  by  the  intended 
OIS  users  and  how  the  target  system  improves  work  patterns.  This  premise  matches 
the  purposes  of  a  PSE  for  knowledge  workers  and  applies  to  the  task  of  evaluating  the 
impact  of  a  PSE. 

A  key  component  of  the  Sassone’s  application  of  the  hedonic  wage  model  is  the  Work 
Profile  Analysis  (also  referred  to  as  Work  Value  Analysis),  which  is  based  on  the 
concept  of  intellectual  specialization  within  knowledge  work.  The  approach  begins 
with  a  baseline  analysis  of  work  patterns  in  an  office  before  introduction  of  an  OIS. 
After  the  OIS  is  implemented,  work  patterns  are  resurveyed.  Comparing  the  “before” 
and  “after”  work  pattern  snapshots  provide  the  basis  for  identifying  the  impact  of  the 
system. 

Sassone  explains  the  concept  of  intellectual  specialization  as  follows.  Organizations 
generally  pay  their  personnel  on  the  basis  of  the  intellectual  content  of  the  work  each 
employee  is  capable  of  doing.  Personnel  with  great  experience  are  compensated  at  a 
higher  rate  than  inexperienced  personnel.  Likewise,  employees  with  advanced  degrees 
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are  paid  more  than  those  with  lesser  degrees.  Engineers,  for  example,  are  paid  more 
than  secretaries. 


Consider,  however,  that  employees  do  not  spend  100  percent  of  their  work  time  doing 
the  kind  of  work  their  background  qualifies  them  to  perform.  Knowledge  workers  may 
spend  only  a  small  portion  of  a  work  day  working  in  their  area  of  expertise.  Most 
knowledge  workers  exert  much  of  their  work  effort  doing  tasks  that  could  be  delegated 
to  less  skilled,  less  expensive  employees  (Sassone  1992). 

A  Work  Profile  Analysis  can  indicate  how  knowledge  workers  at  various  levels  within 
an  organization  typically  spend  their  time.  By  calculating  the  typical  number  of  hours 
worked  and  the  total  cost  of  each  type  of  work,  the  actual  cost  of  each  type  of 
knowledge  work  within  a  specific  organization  can  be  figured. 

The  steps  in  creating  a  Work  Profile  Analysis  are  to:  (1)  categorize  the  work, 
(2)  survey  the  employees,  (3)  develop  a  matrix  analysis,  (4)  implement  the  system 
under  study,  (5)  resurvey  employee,  (6)  compile  a  second  matrix  analysis,  and 
(7)  compare  the  baseline  and  second  matrices  to  evaluate  the  impact  of  the  information 
system. 

The  purpose  of  work  categorization  is  to  classify  work  performed  by  specific  work¬ 
groups.  This  phase  is  accomplished  by  reviewing  mission  statements,  job  descriptions, 
and  interviewing  (both  management  and  nonmanagement)  personnel.  The  workgroup  s 
tasks  are  decomposed  to  the  lowest  level  to  which  each  can  be  delegated.  Tasks  are 
then  consolidated  to  produce  an  averaged  grouping  of  functions.  The  output  of  this 
step  is  a  survey  instrument. 

In  the  survey  step,  each  workgroup  member  logs  data  that  indicates  how  much  time 
he  or  she  spends  within  each  category  of  work.  In  Sassone’s  studies,  each  knowledge 
worker  recorded  information  hourly  in  response  to  the  question:  “How  many  minutes 
did  you  spend  in  each  of  the  pre-defined  categories  of  work? 

A  matrix  analysis  is  created  by  gathering  data  from  the  surveys  and  compiling  the 
information  into  a  matrix  that  indicates  the  workgroup’s  intellectual  work  distnbution 
by  organization  position.  This  matrix  will  serve  as  a  baseline  for  the  workgroup  under 
study.  For  example,  the  information  gathered  from  the  surveys  (and  initial  interviews) 
is  compiled  into  a  preliminary  matrix  (Table  3)  that  shows  distribution  of  effort  within 
each  staffing  level.  This  information  will  serve  as  a  baseline  for  the  workgroup  under 

study. 
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Table  3.  Baseline  matrix  of  occupation  type  by  work  type  actually  performed. 


Position 

Managerial 

Professional 

Support 

Non-Productive 

Managerial 

30% 

20% 

20% 

30% 

Professional 

5% 

35% 

35% 

25% 

Support 

0 

0 

75% 

25% 

Total 

5% 

15% 

55% 

25% 

Matrix  Analysis  I 

Table  3  shows  that  managers  spend  30  percent  of  their  time  performing  managerial 
work,  (e.g.,  planning,  personnel  work,  budgeting,  upward  reporting),  20  percent  of 
their  time  in  professional  work  (e.g.,  preparing  technical  presentations,  writing 
technical  reports,  technical  analysis),  20  percent  of  their  time  in  support  work  in 
support  tasks  (e.g.,  clerical  tasks,  fixing  hardware  or  software,  filing,  photocopying,  or 
keying  in  data).  The  remaining  30  percent  of  their  time  is  spent  on  nonproductive 
tasks  (e.g.,  looking  for  information,  waiting  on  people  or  equipment,  playing  telephone 
tag,  walking  between  buildings). 

Following  Sassone’s  approach,  the  office  information  system  is  implemented  and 
allowed  to  operate  long  enough  that  all  start-up  effects  are  over.  Once  the  system  is 
in  its  operational  stage  and  system  benefits  can  be  realized,  the  workgroup  is 
resurveyed.  Results  of  this  second  survey  are  compiled  into  the  second  matrix. 
Employees  are  resurveyed  at  that  time  to  produce  a  post-implementation  matrix 
analysis. 

Matrix  Analysis  II 

The  second  matrix  (Table  4)  shows  that  the  overall  distribution  of  time  within  each 
category  of  work  has  changed.  The  implication  is  that  managers  accomplish  more 
higher-level  functions— the  desired  effect.  Assuming  that  managers  and  senior 
professionals  are  paid  higher  salaries  than  support  personnel,  the  organization  can 
realize  an  economic  gain  from  this  shift. 


Table  4.  Baseline  matrix  of  occupation  ty 

pe  by  work  type  actually  performed. 

Position 

Managerial 

Professional 

Support 

Non-Productive 

Managerial 

60% 

20% 

10% 

10% 

Professional 

|  15% 

55% 

15% 

15% 

Support 

0 

0 

80% 

20% 

Total* 

10% 

30% 

45% 

15% 
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The  results  of  the  Work  Profile  Analysis  can  be  used  to  evaluate  the  impact  of  the  OIS. 
The  results  may  also  indicate  the  need  for  additional  support  personnel  or  other 
restructuring  within  the  office.  As  Sassone  has  written,  many  organizations  have 
managers  and  highly-skilled  professionals  who  spend  too  little  time  in  work  requiring 
their  expertise.  Often,  much  of  the  professional’s  time  is  spent  in  work  that  could  be 
delegated  to  lower-paid  employees.  In  many  cases,  restructuring  the  office  or  work¬ 
group  can  correct  this  problem. 

Specifically,  there  are  often  insufficient  numbers  of  secretaries  and/or  clerks  to  handle 
the  volume  of  clerical  work  required.  Hiring  additional  clerical  help  or  otherwise 
restructuring  the  environment  is  usually  a  less  expensive  alternative  than  paying 
higher-skilled  workers  to  perform  these  tasks  (Sassone  1992). 

Work  Profile  Analysis  can  be  accomplished  in  a  PSE  via  an  automated  or  manual  tool. 
USACERL  has  created  a  software  instrument  that  achieves  the  time  logging  phases 
of  the  methodology  (summarized  in  Appendix  C).  The  other  steps,  work  categorization 
and  development  of  the  work  matrix,  must  be  performed  manually.  Alternatively,  the 
entire  process  can  be  performed  manually,  as  in  Sassone’s  studies  (Sassone  1992). 
Appendix  D  to  this  report  gives  instructions  and  a  sample  dictionary  of  work  categories 
derived  for  knowledge  workers  at  an  office  in  Headquarters,  U.S.  Army  Corps  of 
Engineers  (HQUSACE). 


Computation  of  Direct  to  Indirect  Work  Ratio 

A  second  possible  approach  to  evaluating  the  impact  of  a  PSE  computes  the  amount 
of  time  spent  on  direct  vs.  indirect  work.  Direct  work  is  defined  as  those  activities 
required  to  generate  mission-related  products.  Indirect  work  includes  tasks  that 
support  personnel  performing  mission-related  work.  The  concept  of  direct  versus 
indirect  work  originated  in  manufacturing.  In  industry,  direct  work  denotes 
production  labor;  indirect  work  refers  to  labor  that  supports  production.  In  a  factory 
setting,  an  employee’s  work  is  considered  as  either  100  percent  direct  or  indirect  labor. 
If  a  worker  is  in  operations  or  production,  that  employee’s  work  is  classified  as  direct. 
Maintenance  or  clerical  work  is  automatically  considered  indirect. 

The  distinction  between  indirect  and  direct  is  more  difficult  in  knowledge  work. 
White-collar  professionals  can  perform  both  types  of  tasks.  In  a  knowledge  work 
setting,  indirect  work  includes  activities  such  as  photocopying,  upward  reporting, 
personnel  management,  filing,  and  responding  to  requests  for  information.  Both 
classifications  of  work  are  necessary,  but  the  organization  is  best  served  if  most  of  a 
knowledge  worker’s  time  is  spent  on  direct  work. 
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Since  knowledge  workers  may  create  goods  or  services,  the  same  employees  are  often 
responsible  for  many  indirect  assignments,  such  as  upward  reporting,  paperwork, 
distribution  of  information,  and  personnel  management  tasks.  Each  workgroup  within 
an  organization  must  find  the  appropriate  balance  between  direct  and  indirect  work 
(Helton  1993). 

One  advocate  of  this  approach  suggests  that  knowledge  workers  should  strive  to  spend 
approximately  60  percent  of  their  time  in  direct  work.  Helton  recommends  performing 
organizational  alignment,  a  process  in  which  the  mission  or  purpose  of  the  organiza¬ 
tion  is  closely  coupled  with  how  work  time  is  spent.  He  suggests  that  employees  who 
spend  more  time  doing  value-added  work  benefit  both  the  organization  and  them¬ 
selves.  Measures  that  focus  work  effort  and  expedite  work  performance  can  result  in 
continuous  improvement  within  the  organization  (Helton  1991). 

Helton’s  approach  would  track  several  trends:  (1)  the  amount  of  direct  work  time, 
(2)  the  amount  of  indirect  work  time,  (3)  the  ratio  of  direct  vs.  indirect  work  time,  and 
(4)  the  ratio  of  direct  to  total  work  time.  Comparison  of  the  ratio  of  direct  vs.  indirect 
work  prior  to  and  following  the  introduction  of  a  PSE  may  yield  a  useful  indicator  of 
the  system’s  effectiveness. 

Helton’s  approach  does  not  emphasize  the  industrial  engineering  focus  on  measuring 
activities,  but  rather  concentrates  on  direct  work  and  speeding  up  work  processes.  He 
states  that  direct  work  adds  value  to  an  organization  by  expediting  the  accomplish¬ 
ment  of  the  business-related,  tangible  outputs.  By  emphasizing  direct  work  and 
therefore  the  mission  of  the  organization,  knowledge  workers  are  encouraged  to  spend 
their  time  and  effort  on  strategic  business  objectives. 

To  apply  these  concepts  to  the  evaluation  of  a  PSE,  the  implementation  team  that 
collects  process  information  prior  to  introduction  of  a  PSE  can  be  tasked  to  identify 
situations  where  direct  versus  indirect  measures  are  easily  discernible.  A  workgroup 
with  one  major  purpose  is  a  likely  candidate  for  calculation  of  direct-indirect  work 
ratios. 

For  example,  an  organization’s  travel  office  has  a  clear  mission:  to  make  travel 
arrangements  for  organizational  personnel.  If  the  travel  office  agrees  that  the  number 
of  travel  orders  processed  quarterly  is  a  good  measure  of  their  productivity,  this  data 
should  be  targeted  for  pre-  and  post-implementation  collection.  The  implementation 
team  should  identify  all  tasks  that  pertain  to  the  production  of  travel  orders,  which 
will  serve  as  the  basis  of  the  direct  work  figure.  The  team  must  ascertain  how  many 
staff  hours  are  spent  on  work  related  to  travel  orders  before  the  introduction  of  the 
PSE.  Any  staff  time  spent  on  work  not  related  to  the  production  of  travel  orders  is 
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figured  as  indirect  work.  The  direct  versus  indirect  work  ratio  can  be  calculated  as  the 
percentage  of  staff  time  spent  on  travel  order  production  relative  to  the  percentage  of 
staff  time  spent  on  other  work. 

A  PSE  installation  generally  requires  an  implementation  team  that  is  responsible  for 
collecting  process  information  for  the  site.  The  implementation  team  should  examine 
tasks  that  comprise  indirect  work  as  possible  Process  Improvement  Possibilities 
(PIPs).  Indirect  work  tasks  may  be  PIP  candidates  that  should  be  eliminated, 
automated,  streamlined,  or  subjected  to  other  process  re-engineering  activities. 

After  identifying  direct  work  tasks  and  collecting  the  number  of  hours  spent  on  direct 
work,  calculate  the  percentage  of  direct  work  time  (staff  hours  spent  on  direct  work 
divided  by  total  staff  hours).  Likewise,  figure  the  indirect  work  percentage  (staff  hours 
spent  on  indirect  work  divided  by  total  staff  hours).  A  direct  to  indirect  ratio  of  1.5  to 
1  has  been  suggested  as  a  reasonable  target  for  most  professional  organizations 
(Helton  1991). 

The  post-implementation  measurement  is  taken  after  the  PSE  is  fully  implemented. 
After  the  staff  has  been  completely  trained  and  the  start-up  costs  of  initiating  a  new 
system  have  been  realized,  the  direct  vs.  indirect  work  ratio  should  be  recalculated  to 
determine  if  the  ratio  has  improved. 


Time  Saved  Times  Salary 

The  Time  Saved  Times  Salary  (TSTS)  technique  examines  labor  efficiency  gains  that 
can  be  attributed  to  the  implementation  of  a  PSE.  The  approach  is  straightforward, 
gains  in  efficiency  are  multiplied  by  labor  cost. 

For  example,  if  a  task  that  took  4  hours  to  perform  before  implementing  a  PSE  can 
completed  in  1  hour  after  system  implementation,  then  a  time  savings  of  3  hours  per 
task  performance  can  be  calculated.  If  the  task  is  required  monthly,  time  savings  of 
3  hours  per  month  or  36  hours  yearly  is  realized.  To  calculate  an  annual  TSTS  figure, 
36  hours  is  multiplied  by  the  hourly  salary  of  the  knowledge  worker  who  normally 
performs  the  task. 

Estimates  of  task  duration  must  be  recorded  before  system  implementation  for  the 
TSTS  technique.  Documentation  of  task  durations  should  be  the  responsibility  of  the 
implementation  team  that  collects  process  information.  It  is  important  to  ask 
personnel  who  actually  do  the  task  how  long  performance  takes.  Ask  task  performers 
to  differentiate  between  elapsed  time  and  time  actually  spent  on  the  task,  from  start- 
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to-finish.  A  change  in  task  performance  can  affect  either  or  both  figures.  Interview 
personnel  to  determine  volume  (the  number  of  times  a  targeted  activity  occurs  within 
a  given  time  frame)  and  repetitions  (the  frequency  with  which  a  process  is  performed). 
These  questions  must  be  asked  prior  to  system  implementation  to  get  accurate  task 
durations.  Once  the  task  has  been  automated  or  the  process  streamlined,  any 
estimates  of  duration  will  be  less  reliable.  Intuitively,  the  duration  of  a  task  is  easier 
to  recall  before  changing  the  task. 

An  additional  application  of  this  data  addresses  the  selection  of  PIPs.  Tasks  that  have 
long  durations,  frequent  iterations,  and/or  high  volume  are  high-priority  candidates 
for  PIPs.  These  are  tasks  that  may  yield  a  high  pay-off  in  efficiency  gains  and 
therefore  should  be  examined  for  automation,  streamlining,  or  other  process 
improvements. 

The  final  step  in  TSTS  is  taken  after  the  PSE  is  fully  implemented.  Some  time 
following  the  introduction  of  the  PSE,  after  the  start-up  costs  of  initiating  a  new 
system  have  been  realized,  post-implementation  task  performance  data  must  be 
collected.  Tasks  that  were  targeted  as  PIPs  should  be  examined  to  determine  if 
(1)  duration,  (2)  frequency,  and/or  (3)  volume  have  changed  following  the  PSE 
implementation.  These  data  can  be  collected  via  interview  or  survey.  Alternatively, 
the  PSE  can  be  constructed  to  collect  the  required  data. 


Activity  Based  Costing 

Activity  Based  Costing  (ABC)  is  a  cost  accounting  method  that  attempts  to  allocate  the 
actual  cost  of  providing  a  service  or  producing  a  product.  ABC  differs  from  traditional 
accounting  practices  that  allocate  all  indirect  costs  through  somewhat  arbitrary 
accounting  rules.  Traditional  accounting  techniques  generally  link  all  overhead  costs 
to  products  or  services  on  the  basis  of  direct  labor  costs. 

According  to  conventional  accounting  rules,  a  service  that  generates  10  percent  of  the 
total  direct  labor  costs  for  an  organization  would  also  be  allocated  10  percent  of  all 
overhead  costs.  ABC  attempts  to  allocate  costs  to  the  services  or  products  that 
generate  these  costs.  ABC  was  derived  from  a  manufacturing  model  that  defines 
production  as  a  set  of  predefined  activities.  The  activities  consume  resources,  which 
generate  costs.  By  allocating  a  product  (or  service)  to  a  set  of  activities  with  an 
incumbent  set  of  resources  and  costs,  a  realistic  cost  of  generating  the  product  or 
service  can  be  calculated. 
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The  following  example  illustrates  the  ABC  approach  (Liggett,  Trevino,  and  Lavelle 
(1992).  Consider  a  company  that  produces  two  products:  gadgets  and  widgets.  Four 
employees  are  responsible  for  performing  all  work.  Two  employees  spend  100  percent 
of  their  time  in  assembling  the  components  required  to  produce  the  gadgets  and 
widgets.  The  other  two  employees  spend  70  percent  of  their  time  inspecting  the 
component  parts  and  finished  products,  and  30  percent  of  their  time  in  material 
handling. 

Each  employee  costs  the  company  $10  per  hour,  including  all  fringes.  During  the 
course  of  1  year,  1.3  million  components  are  received  and  assembled  into  finished 
products.  One  million  tests  are  performed  to  inspect  the  parts  and  finished  products. 
Components  are  moved  from  storage  into  the  assembly  stations  in  batches  of  50.  The 
distance  traveled  for  retrieval  of  each  gadget  component  is  250  ft;  for  widgets,  the 
distance  traveled  is  40  ft  (1  ft  =  0.305  m).  Gadgets  require  four  components;  widgets 
require  six.  Seven  tests  are  conducted  for  each  gadget  produced;  two  tests  are 
conducted  for  each  widget.  Annual  gadget  production  is  100,000  units;  annual  widget 
production  is  150,000  units. 

Using  this  data,  unit  cost  values  for  gadgets  and  widgets  can  be  constructed  using  the 
ABC  paradigm.  There  are  three  activities  in  producing  the  two  products:  assembly, 
inspection,  and  material  handling.  The  cost  drivers  for  each  activity  are:  (1)  assembly 
-  number  of  components  assembled,  (2)  inspection  -  number  of  tests  conducted,  and 
(3)  materials  handling  -  number  of  feet  the  components  are  moved. 

Tables  5  through  9  give  a  summary  analysis  of  the  process.  Table  5  links  the  products 
to  the  activities  through  cost  drivers.  Table  6  links  activities  to  resources  and  the 
resources  to  costs.  Table  7  provides  the  calculation  of  the  unit  cost  of  each  activity. 
For  example,  the  average  cost  of  assembling  components  is  $40,000  divided  by  1.3 
million,  or  $0.0308  each.  Table  8  shows  the  unit  cost  of  producing  each  component. 
For  example,  the  unit  cost  of  producing  a  gadget  is  $0.4073.  Table  9  allows  a 
comparison  of  unit  costs  as  calculated  using  ABC  with  unit  costs  as  figured  using 
traditional  cost  accounting  rules. 


Table  5.  Costs  of  products  as  related  to  activities. 


Cost  Drivers 

Products 

Assembly 

Inspection 

Material  Handling 

Gadgets 

4 

7 

4  X  250/50  =  20 

Widgets 

6 

2 

6  X  40/50  =  4.8 
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Table  6.  Activities  related  to  resources  and  costs. 


Activities 

Total  Hrs/yr 

Cost/hr 

Cost/yr 

Assembly 

2  x  40  x  50  =  4000 

$10.00 

$40,000 

Inspection 

2  x  40x50x70%  =  2800 

$10.00 

$28,000 

Material  Handling 

2  x  40x50x30%  =  1200 

$10.00 

$12,000 

Table  7.  Determining 

the  unit  cost  of  activities. 

Activities 

Cost/yr 

No.  of  Drivers 

Cost  Per  Driver 

Assembly 

$40,000 

1 .3  million  components 

$.0308/component 

Inspection 

$28,000 

1.0  million  tests 

$.0280/test 

Material  handling 

$12,000 

2.72  million  feet 

$.0044/foot 

Table  8.  Calculating  the  unit  cost  of  production  via  ABC. 


Assembly 

Inspection 

Material  Handling 

ABC  Unit  Cost 

Product 

No. 

Cost 

No. 

Cost 

No. 

Cost 

$0.4073 

Gadget 

4 

$0.0308 

7 

$.0280 

20 

$0.0044 

$0.2618 

Widget 

6 

$0.0308 

2 

$.0280 

4.8 

$0.0044 

$0.2618 

Table  9.  Calculating  unit  costs  of  production  using  conventional  cost  accounting. 


Product 

Total  Direct 
Labor  Costs 

%  of  Direct 
Costs 

Overhead 

Allocation 

Total  Production 
Costs 

Conventional 
Unit  Costs 

%  Error  in  Unit 
Costs 

Gadgets 

$12,300 

30.75% 

$  12,300 

$  24,600 

$.2460 

-39.6% 

Widgets 

$  27,700 

69.25% 

$27,700 

$  55,400 

$.3693 

+41.1% 

Totals 

$  40,000 

100% 

$  40,000 

$  80,000 

In  the  latter  case,  the  assembly  costs  of  gadgets  and  widgets  are  viewed  as  direct  costs; 
the  inspection  and  material  handling  costs  are  treated  as  indirect.  The  indirect  costs 
would  be  allocated  to  gadgets  and  widgets  in  proportion  to  the  direct  costs  of  each 
product.  The  amounts  that  would  result  from  the  conventional  cost  accounting 
approach  are  calculated  in  Table  9.  Note  that  the  last  column  indicates  traditional 
cost  accounting  and  yields  an  error  of  approximately  40  percent  in  allocating  the  unit 
costs  of  the  two  products. 
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ABC  has  been  applied  to  the  service  industry  in  an  effort  to  give  managers  a 
framework  for  making  sound  business  decisions  by  identifying  all  the  costs  associated 
with  providing  a  particular  service.  Using  ABC,  organizations  can  more  clearly  see  the 
true  costs  of  products  and  services.  Organizations  can  use  this  information  to  make 
decisions  that  improve  the  profitability  of  their  operations.  ABC  can  be  applied  to  the 
cost  justification  of  a  performance  support  environment  in  certain  specific  situations. 
The  ABC  approach  can  be  useful  in  production  offices  and  in  structured  knowledge 
work  environments  where  tasks  are  essentially  repetitive. 

Consider  a  production  office  where  the  primary  work  requirement  involves  repetitively 
performing  certain  processes,  such  as  preparing  reports,  preparing  budgets,  and 
collecting  data  from  other  personnel.  The  tasks  may  be  repeated  on  a  daily,  weekly, 
monthly,  yearly,  or  other  periodic  basis. 

Each  process  is  initiated  by  a  specific  driver,  such  as  a  request  from  management,  the 
approach  of  the  due  date  for  a  weekly  report,  a  data  call,  or  appointment  to  a 
committee.  Each  process  is  composed  of  tasks,  such  as  entering  data  into  a  spread¬ 
sheet,  sending  email  messages,  attending  meetings,  preparing  documents,  scheduling 
meetings,  or  making  phone  calls.  For  example,  Table  10  shows  the  application  of  the 
ABC  technique  to  the  preparation  of  a  weekly  report  that  indicates  how  60  knowledge 
workers  charged  their  time  to  projects. 

The  ABC  breakdown  of  processes,  tasks,  and  costs  provides  information  that  would  be 
useful  both  in  building  the  database  of  processes  and  in  selecting  PIPs.  However,  the 
cost  of  the  tasks  that  comprise  each  process  is  difficult  and  expensive  to  assess.  A 


Table  10.  ABC  applied  to  knowledge  work  task. 


Element 

Example 

Driver 

Weekly  requirement  to  report  division’s  staff  hours  (by  project)  for  labor  accounting  system 

Task 

Prepare  Labor  &  Time  Sheet  for  60  knowledge  workers 

Activities 

1 .  Send  email  request  for  report  of  time  spent  on  each  project  for  weekly  reporting  period. 

2.  Collect  delinquent  reports. 

3.  Check  all  reports  for  accuracy  &  completeness. 

4.  Contact  individuals  with  inaccurate  or  incomplete  reports. 

5.  Enter  data  from  scrubbed  reports  into  system. 

Time  spent  on  task 

Activity  1: 

0.10  hour 

(per  week) 

Activity  2: 

0.40  hour 

Activity  3: 

1.10  hour 

Activity  4: 

0.30  hour 

Activity  5: 

0.60  hour 

Total: 

2.50  hours  per  week 

Implicit  cost  of  task 
(per  year) 

$12.95*130 

=  $  1,683.50  labor  cost  X  total  No.  of  hours  =  annual  cost 
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manual  or  automated  time  logging  instrument  could  be  constructed  to  help  gather  the 
implicit  costs  of  each  task. 

The  time  and  expense  of  collecting  this  information,  however,  is  probably  not 
warranted  unless  the  workgroup  being  analyzed  is  both  structured  and  stable,  i.e.,  if 
the  tasks  are  always  done  using  the  same  procedures,  and  if  the  processes  are  not 
likely  to  be  changed  for  several  months.  If  these  conditions  are  not  met,  the  resources 
necessary  to  implement  ABC  are  likely  to  be  greater  than  the  benefit  gained. 

ABC  is  also  recommended  as  an  evaluation  technique  in  those  cases  where  both 
Integrated  Definition  (IDEF)  process  modeling  and  ABC  have  been  performed  as  part 
of  other  initiatives.  The  Department  of  Defense  (DOD)  has  recommended  the  use  of 
both  IDEF  and  ABC  as  part  Functional  Process  Improvement  efforts  in  the  Defense 
Information  Management  Program  (1993). 

If  ABC  has  been  implemented  for  workgroups  where  a  performance  support 
environment  (PSE)  is  being  introduced,  the  results  can  be  used  to  identify  the 
processes  that  consume  the  largest  amount  of  labor  resources.  The  most  costly 
processes  should  be  examined  as  likely  PIPs,  as  mentioned  previously.  These  same 
processes  may  warrant  further  data  collection  after  the  PSE  has  been  fully  imple¬ 
mented.  This  post-implementation  data  can  be  done  using  either  another  technique 
such  as  Work  Profile  Analysis  or  Time  Saved  Times  Salary,  or  by  re-application  of 
ABC. 

Quality  Assessment 

Improved  quality,  along  with  enhanced  employee  performance,  is  a  key  benefit 
acclaimed  by  performance  support  systems  (Gery  1991).  Along  with  any  performance 
impacts  an  organization  tracks,  changes  in  quality  should  also  be  monitored.  The 
desire  for  improved  quality  is  one  of  the  top  six  reasons  that  organizations  invest  in 
information  technology,  where  the  top  six  reasons  for  investing  in  information 
technology  are  to: 

1.  Improve  infrastructure  for  communications  and  data  handling 

2.  Meet  mandated  requirements 

3.  Reduce  costs 

4.  Provide  new  products 

5.  Improve  quality 

6.  Facilitate  major  strategic  repositioning. 
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Paradoxically,  organizations  report  difficulty  in  measuring  the  impact  of  IT  on  quality. 
Further,  the  impact  of  the  technology  investment  is  largely  meaningless  without  a 
valid  metric  of  the  quality  of  the  resulting  output  (National  Research  Council  1994). 

Service-oriented  organizations  have  provided  leadership  in  developing  tools  and 
methods  that  track  customer-oriented  measures  of  quality.  These  tools  were  developed 
in  response  to  the  strong  positive  correlation  between  costs  and  the  quality  of  service. 
Reduction  of  errors  in  producing  a  service  reduces  both  coordination  costs  and  rework, 
as  well  as  customer  complaints. 

Organizations  implementing  a  PSE  are  encouraged  to  develop  and  use  customer- 
oriented  quality  metrics  for  similar  reasons.  The  tools  used  by  private  sector  service 
organizations  are  largely  applicable  for  use  within  government  environments.  For 
example,  some  methods  used  by  service  organizations  are:  focus  groups,  user  groups, 
quality  circles,  process  action  teams,  pilot  tests,  surveys,  sampling,  interviews, 
observation,  and  other  quality  management  tools  (Deming  1986).  Each  of  these  may 
be  useful  in  assessing  quality  within  a  specific  setting. 

Minimally,  methods  should:  (1)  collect  customer  feedback,  (2)  collect  supervisory 
feedback,  and  (3)  provide  opportunities  for  self-assessment.  This  information  can  be 
collected  fairly  inexpensively,  using  methods  such  as  interviews,  surveys,  and/or  tools 
built  into  the  system  itself. 

For  example,  KWS  will  use  surveys  and  on-line  tools  to  collect  quality-related  data. 
Customers  of  KWS  users  will  be  surveyed  to  ascertain  their  level  of  satisfaction  with 
the  timeliness,  completeness,  and  general  quality  of  the  products  generated  using 
KWS.  KWS  users  will  be  surveyed  via  periodic  electronic  mail  surveys  about  their  own 
productivity.  Supervisors  of  KWS  users  will  likewise  be  surveyed  to  assess  the 
productivity  and  quality  of  work  performed  by  their  subordinates  who  use  the  system. 
Finally,  KWS  users  will  have  the  capability  to  summon  an  on-line  tool  that  allows 
comments  on  the  usefulness  of  specific  KWS  features. 

The  information  collected  will  be  used  to:  (1)  assess  the  perceived  quality  of  the  work 
performed  by  KWS  users,  and  (2)  gather  information  on  the  perceived  usefulness  of 
system  functionality.  The  first  category  of  information  will  be  collected  prior  to  and 
following  KWS  implementation. 

Tools  for  collecting  data  about  quality  issues  have  been  thoroughly  detailed  in  the 
Total  Quality  Management  (TQM)  literature  (Brassard  1988,  Deming  1986,  Cleary 
1993).  Figure  4  lisf "  several  to.„ls  described  in  TQM  literature.  The  tools  are  grouped 
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into  “problem  identification”  or  “problem  analysis”  categories.  Several  tools  are  useful 
for  both  types  of  problem  solving. 

Organizations  implementing  a  PSE  are  encouraged  to  assess  quality  along  with  the 
particular  performance  measures  they  choose  to  track.  Quality-related  data  from 
customers,  PSE  users,  and  from  supervisors  of  PSE  personnel  can  provide  important 
feedback  about  the  strengths  and  weaknesses  of  the  PSE.  Table  11  summarizes  the 
PSE  evaluation  techniques,  the  environments  where  they  might  best  be  applied,  and 
the  advantages,  disadvantages,  and  relative  cost  of  each  technique. 
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Table  11.  PSE  evaluation  techniques. 


Technique 

Recommended  Environment 

Pro 

Con 

Cost* 

Work  Profile 
Analysis 

All 

Complements  PSE 
purpose;  Can  be 
automated 

Data  intensive; 

Time  consuming 

Low-med 

Direct  vs  Indirect 

Production  Office 

KW** 

Intuitive;  Identifies 
tangential  activities 

Data  intensive;  Time 
consuming 

Low-med 

TSTS 

All 

Easy  to  implement 
Intuitive;  Attractive 
to  management 

May  measure  the 
wrong  attribute 

Low 

ABC 

Production  Office  KW  ** 

Identifies  costs 

Data  intensive;  Time 
consuming 

Med-high 

Workflow 

Production  Office 

Systematic;  Good 
analysis  tool 

Time  consuming 

Med-high 

Quality 

All 

By  definition, 
measures  what’s 
important 

Can  be  time 
consuming 

Low-med 

*  Costs  can  be  reduced  if  initial  information  is  collected  as  part  of  process  building  performed  by  the  PSE 
implementation  team. 

**  Also  appropriate  in  knowledge  work  environments  where  processes  are  stable  and  structured.  Note  that  many 
DOD  organizations  have  performed  ABC  as  part  of  other  process  modeling  initiatives. 
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5  Conclusion  and  Recommendations 


A  broad  review  of  relevant  literature  shows  that  no  single  tool  can  effectively  measure 
and  evaluate  an  activity  as  complex  and  intangible  as  knowledge  work,  or  a 
performance  support  environment  (PSE)  for  knowledge  workers.  This  study  concludes 
that  a  “toolkit”  of  five  evaluation  techniques,  each  applicable  to  a  specific  workgroup 
setting,  may  best  assess  the  feasibility  and  usefulness  of  a  PSE: 

1.  Work  Profile  Analysis 

2.  Direct  to  Indirect  Ratio 

3.  Time  Saved  Times  Salary  (TSTS) 

4.  Activity  Based  Costing  (ABC) 

5.  Quality  Assessment. 

The  first  step  in  selecting  the  appropriate  tools  for  evaluation  is  to  categorize  the  type 
of  knowledge  work  performed  within  the  environment.  The  potential  PSE  site  should 
be  analyzed  for  four  composite  attributes:  (1)  complexity,  (2)  volume  per  job,  (3)  time 
per  job,  and  (4)  repetition. 

A  workgroup  should  first  be  categorized  as  either  a  knowledge  work-intensive  (KW) 
professional  environment,  or  a  production  office.  A  professional  office  is  one  whose 
primary  function  requires  professional  level  workers,  where  the  work  is  not 
substantially  repetitive,  and  where  the  clerical  work  is  performed  in  support  of 
professional  work.  A  production  office  is  an  office  whose  primary  function  is  the 
performance  of  a  small  number  of  repetitive,  clerical-level  tasks,  e.g.,  claims 
processing,  order  entry,  and  call  centers. 

Once  the  workgroup  is  categorized,  an  appropriate  evaluation  tool  should  be  selected. 
The  Information  Economics  (IE)  cost-benefit  analysis  is  recommended  as  the  first  step 
in  deciding  which  PSE  impacts  to  evaluate.  It  is  also  recommended  that  a  quality 
assessment  be  done  in  conjunction  with  any  other  evaluation  tool. 
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Appendix  A:  Taxonomy  of  KWS  Benefits  and 
Capabilities 

KWS  Implementation  Scoresheet 

The  following  Scoresheet  (Figure  Al)  is  divided  into  four  blocks  to  facilitate  the 
discussion  of  how  to  complete  and  utilize  it.  Each  block  has  a  specific  function  that 
is  detailed  in  Figures  A2  to  A5. 

BLOCK  1  The  value  codes  appropriate  to  each  evaluation  class  are  entered  in 

the  “value  code”  column.  The  value  codes  are  detailed  in  block  4.  For 
example,  if  improving  performance  is  important  to  you,  enter  a  “4”  in 
the  code  column  next  to  that  class  (Figure  A2). 

BLOCK  2  Each  benefit  category  is  related  to  each  value  class  by  entering  the 

appropriate  relation  code  in  the  corresponding  columns.  The  relation 
codes  are  detailed  in  block  4.  For  example,  if  you  determine  that 
reduced  rework  is  strongly  related  to  Performance  Improvements, 
enter  a  “4”  in  the  #2-value-column  of  that  category  [B]  (Figure  A3). 
The  “Ind  Benefit  Scores”  column  on  the  right  side  of  block  4,  is  where 
the  sum  of  the  products  for  each  category  is  entered.  Figure  3  shows 
an  example  where  a  “16”  is  entered — a  value  of  4  times  a  relation  of  4. 
Figure  4  shows  a  more  complete  example. 

BLOCK  3  The  sum  of  the  individual  scores  from  block  2  is  entered  in  the  space 
to  the  right  of  “SCORE  FOR  KWS  IMPLEMENTATION”  (Figure  5). 

BLOCK  4  The  codes  used  to  rate  value  classes  and  benefit  categories  are  ex¬ 

plained  in  this  block.  These  are  not  hard  quanitiative  values,  but  are 
“fuzzy”  values,  i.e.,  your  interpretation  of  the  values  and  benefits  of 
these  items  to  the  process  being  examined.  The  definitions  given  are, 
therfore,  purposely  ambiguous  so  that  they  can  be  applied  to  any 
situation. 
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BLOCK  4 

EXPLANATIONS  OF  CODES 

Relation  Codes 

Value  Codes 

0 

No  Relation 

0 

Not  Relevant 

1 

Minimal  Relation 

1 

Not  Important 

2 

Some  Relation 

2 

Little  Importance 

3 

Moderate  Relation 

3 

Moderate  Importance 

4 

Strong  Relation 

4 

Important 

5 

Absolute  Relation 

5 

Very  Important 

Figure  A1.  Scoresheet  for  KWS  implementation. 


Figure  A2.  First  block  of  KWS  implementation  scorecard. 


BLOCK  2 


KWS  BENEFITS 


numbered  value 
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Value 

#  Name 

1  ENHANCED  ROI 

2  PERFORMANCE  IMPROVEMENTS 

3  STRATEGIC  MATCH 

4  STRATEGIC  IS  ARCHITECTURE 

5 

6 

Ind 

Benefit 

Benefit 

Ltr.  Name 

Scores 

A  Improved  Effectiveness 

0 

B  Reduced  Rework 

33 

C  Improved  Efficiency 

0 

D  Improved  Focus 

0 

E  Work  Elimination 

0 

F 

0 

G 

0 

H 

0 

I 

0 

J 

0 

K 

0 

L 

0 
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BLOCK  2 


KWS  BENEFITS 

Relation  to 

Ind 

numbered  value 

Benefit 

Benefit 

1 

2 

3 

4 

5 

6 

Ltr. 

Name 

Scores 

4 

4 

0 

0 

A 

Improved  Effectiveness 

28 

5 

4 

0 

1 

B 

Reduced  Rework 

33 

4 

4 

0 

0 

C 

Improved  Efficiency 

28 

5 

5 

3 

0 

D 

Improved  Focus 

39 

5 

5 

2 

0 

E 

Work  Elimination 

38 

F 

0 

G 

0 

H 

0 

I 

0 

J 

0 

K 

0 

L 

0 

BLOCK  3 

SfORF  FOR  KWS  IMPI  .FMFNTATION  Ut 


Figure  A5.  Sample  completed  benefit  scores. 

Primary  Objectives 

The  primary  objectives  of  the  Knowledge  Worker  System  are: 

•  Enhanced  Knowledge  Worker  Performance 

•  Improved  Quality. 


Benefit  I.  Effectiveness  Improvements 

This  benefit  encompasses  the  capabilities  described  in  benefits  II  -  V. 

Benefit  II.  Rework  Reductions 

Capability:  Process  Model  Documented 

a.  Describes  work  procedures  in  detail 

b.  Disseminates  changes  in  process  to  all  players 
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c.  Facilitates  delegation  of  portions  of  process 

d.  Provides  clear  specification  of  assignments 

Capability:  Institutional  Knowledge  Captured 

a.  Records  experience/tips/lessons  learned  of  previous  job  occupants 

b.  Trains  new  employees 

c.  Facilitates  job  rotation 


Benefit  III.  Efficiency  Improvements 

Capability:  Shared  Task  Schedule 

a.  Identifies  task  priorities 

b.  Monitors  task  status 

c.  Automates  resource  leveling 

d.  Provides  early  notification  of  deadlines 

e.  Updates  schedule  changes 

f.  Allows  what-if  analysis  of  schedule 

g.  Provides  coordination  between  workgroup  members 

h.  Monitors  delegated  tasks 

Benefit  IV.  Focus  Improvements 

Capability:  Referential  Information  Linked  to  Tasks 

a.  Reduces  time  spent  accessing  references 

b.  Provides  examples  of  previous  submissions 

c.  Frees  time  for  analysis/decisionmaking 

d.  Allows  referential  information  to  be  shared 


Benefit  V.  Work  Elimination 

Capability:  Tasks  Automated  &  Linked  to  Tasks 

a.  Reduces  time  spent  on  repetitive  tasks 

b.  Reduces  time  spent  on  accessing  external  software/systems/databases 

c.  Allows  automation  tools  to  be  shared 
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Appendix  B:  An  Analysis  of  the  Techniques 
and  Application  of  Information  Economics 
toKWS 


Introduction 

Information  economics  (IE)  is  an  effort  to  provide  a  more  complete  means,  when 
compared  to  simple  return  on  investment  (ROI),  of  evaluating  potential  projects  for  a 
company.  IE  is  a  form  of  decisionmaking  and  therefore,  the  evaluation  technique  is 
not  limited  to  evaluating  projects,  but  can  be  used  in  many  decisionmaking  situations. 
The  application  of  this  technique  to  evaluating  potential  implementation  sites  for  the 
Knowledge  Worker  System  (KWS)  would  be  very  useful.  IE  could  provide  an  excellent 
approach  of  determining  which  performance  trends  to  track  for  that  KWS  pilot  site. 
The  following  text  is  an  analysis  of  the  techniques  used  in  Information  Economics  and 
its  extension  to  KWS. 


Analysis  of  IE 
Background 

A  typical  comparison  of  possible  projects  is  based  on  accounting  type  of  information, 
i.e.,  projected  costs,  projected  benefits,  and  return  on  investment.  The  comparison  may 
involve  several  projects  competing  for  resources  or  it  may  simply  be  a  single  project 
competing  with  the  current  situation.  The  problem  with  all  of  this  is  that  it  ignores 
a  number  of  factors  that  may  effect  the  success  of  the  projects)  simply  because  these 
factors  do  not  lend  themselves  to  dollar  quantification.  IE  provides  a  methodology  for 
including  nonquantifiable  factors  in  the  analysis  of  projects.  (For  a  detailed  presenta¬ 
tion  see  Parker,  Benson,  Trainor  1988.) 

It  is  important  to  note  that  IE  attempts  to  make  explicit  some  of  the  biases  implicitly 
applied  in  evaluating  alternative  projects.  For  example,  the  organization’s  aversion 
to  risk  may  be  recognized  as  a  factor  in  rating  projects.  Different  organizations  may 
come  to  different  conclusions  when  evaluating  the  same  project. 
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The  most  important  factors  in  evaluating  projects  are  recognized  and  given  relative 
weights.  A  firm  very  interested  in  ROI  may  use  a  weight  of  10  versus  a  weight  of  2  for 
competitive  response.  Then  each  project  is  ranked  for  each  factor.  In  the  case  of  ROI 
and  other  quantitative  measures,  actual  numbers  can  be  applied.  The  weights  for  each 
factor  are  applied  to  each  project’s  rankings  for  the  factors  and  a  total  score  is 
determined.  A  comparison  of  these  scores  can  give  a  good  picture  of  the  project(s)  that 
are  best  suited  to  this  organization’s  vision. 

The  application  of  IE  can  be  observed  in  a  “tool”  developed  by  Oracle  Corp.  called  CB- 
90  -  Cost  Benefit  for  the  Nineties.  CB-90  breaks  the  analysis  down  into  three  factors: 
Tangible  cost/benefit  analysis,  Intangible  cost/benefit  analysis,  and  Intangible  risk 
analysis  (Semich  1994).  The  latter  two  can  be  further  subdivided  into  business  and 
technical  groupings  (Cox  1993).  These  two  articles  with  another  (Pastore  1992) 
provide  a  valuable  insight  into  the  application  of  the  IE  theory. 


The  IE  Methodology 

Information  Economics  focuses  on  value  rather  than  the  more  limited  concept  of 
benefit.  It  should  be  noted  that  much  of  the  terminology  used  in  IE  represents  an 
application  to  the  nongovernmental  organization.  This  does  not  decrease  its 
applicability  to  governmental  structures.  It  simply  means  that  some  adjustments  will 
be  required.  IE  uses  six  classes  of  value  summarized  as  follows: 

1.  Enhanced  ROI  —  Like  a  standard  return  on  investment  but  expanded  to  include 
additional  methods:  value  acceleration,  value  linking,  value  restructuring,  and 
innovation  valuation. 

2.  Strategic  Match  —  This  is  a  measure  of  how  closely  aligned  the  project  is  to  the 
organization’s  strategic  goals. 

3.  Competitive  Advantage  —  Estimates  the  degree  to  which  the  project  provides  an 
advantage  in  the  marketplace.  This  can  be  viewed  as  an  improvement  in  the 
product  or  service  of  the  organization  or  of  a  sharpening  of  the  focus  of  its  vision. 

4.  Management  Information  —  Management  needs  information.  The  value  of  the 
information,  or  the  improved  information,  that  the  project  is  expected  to  provide 
is  the  factor  here.  The  more  essential  to  the  functioning  of  the  organization  that 
the  project’s  information  is  the  more  valuable  it  may  be. 
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5.  Competitive  Response  —  This  is  an  estimate  of  the  consequences  of  not 
implementing  the  project.  This  can  be  viewed  as  the  nonmarketing  consequences 
of  inactivity.  An  organization  that  appears  unable  to  handle  its  work  effectively 
may  no  longer  have  that  work  to  do  or  it  may  find  itself  reorganized  into  some 
other  organization. 

6.  Strategic  IS  Architecture  —  The  implicit  assumption  in  this  value  is  that  there 
is  some  strategic  plan  for  the  information  systems  of  the  organization.  The 
measure  here  is  of  how  the  project  fits  into  and/or  complements  that  overall  plan. 
In  the  absence  of  a  plan  it  may  be  measured  as  to  how  well  it  lays  the  ground¬ 
work  for  such  a  plan. 

Each  of  these  value  classes  is  assigned  a  weight  for  the  organization  in  which  the  IE 
analysis  is  occurring.  Eventually  every  proposed  project  will  be  evaluated  in  each 
value  class  for  its  effect  and  then  a  summarized  score  is  calculated  for  each  project. 


IE  applied  to  KWS 
Background 


Typically,  with  IE,  an  organization  starts  with  a  vision  and  proceeds  to  establish  the 
criteria  and  their  weights  to  be  used  in  deciding  which  projects  best  fit  into  its  vision. 
In  applying  IE  to  KWS  we  must  start  with  a  set  of  potential  benefits  attributable  to 
KWS  and  then  set  up  a  methodology  for  evaluation  in  differing  environments.  The 
following  illustration  clarifies  this  difference. 

PERFECT  Company  places  great  value  on  never  making  a  mistake  on  a  customer’s 
order.  Their  vision  is  focused  on  this  error  free  goal.  Their  evaluation  of  any 
project  will  weight  heavily  its  affect  on  perfection.  CHEAP  Company  on  the  other 
hand  takes  great  pride  in  being  the  cheapest  price  on  the  market  and  naturally  will 
weight  any  project  heavily  for  its  affect  on  price.  The  typical  IE  evaluation  would 
take  place  in  the  environment  of  the  subject  company  and  would  have  criteria  and 
weights  established  based  on  the  vision  of  that  company.  The  approach  we  must 
make  in  evaluating  KWS  is  to  develop  a  set  of  criteria  in  a  tool  such  that  the  vision 
of  the  implementation  prospect  can  be  accounted  for. 

In  applying  information  economics  methodology  to  KWS  implementation  it  is 
important  to  remember  that  IE  is  used  to  estimate  the  value  of  a  choice  to  an 
organization.  If  IE  allows  you  to  estimate  this  value  before  the  event,  then  it  can  also 
be  applied  to  estimating  the  value  of  the  decision  after  the  event.  As  the  majority  of 
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the  work  will  have  been  accomplished  in  the  initial  estimate,  the  work  involved  in  the 
second  estimate  will  be  greatly  reduced. 

The  specific  means  of  evaluating  the  selected  criteria  is  a  separate  decision.  The  IE 
process  establishes  what  is  deemed  important  by  the  user.  The  appropriate  measures 
and  tools  can  then  be  selected. 

Consider  the  example  mentioned  above.  The  PERFECT  Company’s  overriding  concern 
is  perfection.  Therefore,  it  will  want  to  evaluate  the  level  of  perfection  achieved  after 
the  decision  is  made,  that  is,  if  it  were  to  use  IE  in  the  same  manner  suggested  here. 

Suppose  that  one  of  the  benefits  of  KWS  is  that  it  makes  the  scheduling  more  efficient, 
and  that  one  member  of  the  implementation  team  has  rated  this  as  a  very  important 
benefit.  Part  of  the  determination  of  the  KWS  productivity  for  this  implementor  would 
be  the  improvement  of  scheduling.  Other  factors  would  also  have  to  be  included  in  this 
evaluation  of  productivity,  but  one  measure  that  must  be  taken  is  the  change  in  the 
scheduling  function. 

One  of  the  important  points  of  this  technique  is  that  the  productivity  change  is  based 
on  the  factors  important  to  the  user.  If  the  cost  of  the  change  is  of  relatively  low 
importance  to  the  user,  this  can  be  accounted  for  in  this  technique.  Likewise,  the 
technique  can  account  for  the  user  for  whom  the  costs  are  very  important.  It  could  be 
argued  that  two  users  in  similar  circumstances  might  then  produce  widely  different 
productivity  changes.  The  real  measure  of  the  productivity  change  is  what  the  user 
perceives  and  if  the  user  has  established  a  vision  then  this  perception  should  be  the 
best  measure  of  utility. 

It  is  important  to  note  that  this  technique  ties  the  measure  of  productivity  to  the 
implementor’s  vision.  This  vision  should  reflect  a  concern  about  the  “customer,”  for  as 
Peter  Drucker  (1974)  noted,  the  purpose  of  an  enterprise  is  to  create  a  customer. 


An  Outline  of  the  Technique 

To  use  IE  techniques  to  evaluate  a  specific  project— the  implementation  of  KWS — we 
must  first  establish  the  benefits  of  KWS.  The  following  section  discusses  a  set  of 
potential  benefits. 

Our  list  of  benefits  is  something  we  take  to  each  potential  implementation  site.  At 
each  site  we  must  then  establish  the  values  of  the  organization.  This  second  step 
allows  us  to  plug  these  figures  into  a  worksheet  and  produce  a  rating  for  the 
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implementation  of  KWS  at  that  site.  As  there  is  no  alternative  project(s)  to  compare 
KWS  to  simply  compare  it  to  the  status  quo.  Of  course,  if  KWS  is  implemented,  we 
will  later  compare  the  post-implementation  and  the  pre-implementation  scores  to 
establish  the  productivity  change.  The  score  for  the  status  quo,  however,  is  not 
necessarily  zero,  although  it  could  be.  The  status  quo  may  have  a  very  negative  score 
(in  those  situations  where  collapse  is  imminent).  It  could  have  a  significantly  positive 
score — in  those  situations  where  operations  are  running  well.  The  comparison  would 
indicate  situations  where  the  impact  of  KWS  would  be  significant  and  minor. 

The  evaluation  tool  then  starts  with  a  group  of  established  benefits.  The  second  step 
is  to  determine  the  weights  to  be  applied  to  the  six  values  of  IE  and  to  score  the  KWS 
benefits  for  the  organization.  This  process  must  be  done  by  the  organization  itself. 


The  Benefits  of  KWS 

The  benefits  of  KWS  can  be  organized  in  many  different  ways.  One  approach  is  to 

separate  the  benefits  into  three  broad  categories: 

1.  Knowledge  Oriented  Benefits  -  An  assortment  of  tools,  data,  and  resources  that 
are  used  in  the  performance  of  work.  In  reference  to  KWS  the  access  to  these  is 
automated  and  therefore  their  utilization  is  both  more  efficient  and  effective. 

2.  Control  Oriented  Benefits  —  Those  aspects  of  KWS  that  allow  both  the  individual 
and  management  to  be  aware  of  the  status  of  work  and  to  respond  in  a  timely 
manner  to  that  status. 

3.  Action  Oriented  Benefits  -  When  work  is  actually  performed,  there  may  be  many 
repetitive  tasks  that  occupy  significant  portions  of  the  performer’s  time. 
Automating  or  organizing  these  tasks  via  KWS  has  a  benefit  in  excess  of  stand 
alone  improvement. 


Knowledge 

•  Reduce  external  references  for  how-to-do-it  information 

How-to-do-it  information  will  exist  on  the  system.  Formal  rules  and  manuals 
will  be  accessible  along  with  learned-on-the-job  information.  Both  new  and 
experienced  workers  will  be  able  to  perform  complex  tasks  with  less  references 
to  external  sources. 
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•  Examples  automatically  available 

Files  and  data  associated  with  particular  tasks  will  be  accessible  via  KWS.  This 
means  that  previous  examples  of  a  task  will  be  easily  available  for  reference  and 
that  the  processes  currently  being  utilized  will  be  accessible.  This  reduces  the 
time  spent  looking  for  information  and  it  increases  the  accuracy  of  the  work 
performed. 

•  Related  information  will  automatically  be  accessible 

Information  associated  with  a  task  will  be  kept  associated  with  that  task.  This 
reduces  the  time  spent  looking  for  missing  information. 

•  Procedural  knowledge  will  automatically  updated 

Procedures  will  be  easier  to  document.  The  procedures  can  grow  from  the  actual 
work  being  performed. 

Control 

•  Priorities  will  be  updated  automatically 

Individuals  will  be  able  to  see  what  their  priorities  are  for  the  day,  week,  and 
further  into  the  future.  They  will  therefore  be  able  to  concentrate  on  the  higher 
priority  items. 

•  Prior  work  will  be  referenced  via  the  system 

Individuals  will  be  able  to  see  what  they’ve  done  in  the  past. 

•  Managers  can  optimize  for  changing  situations 

Managers  can  optimize  the  abilities  of  the  people  available  by  reassigning  work 
and  or  priorities  as  situations  change. 

•  Scheduling  will  be  automated 

Annual  leave,  training,  and  other  such  activities  can  be  scheduled  in  less  time 
and  more  effectively. 

•  The  ability  to  function  as  a  group  is  enhanced 

Individuals  can  perform  as  members  of  the  group  by  accessing  schedule  and 
other  information  and  then  acting  on  it. 
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Action 

•  Repetition  will  be  reduced 

Repetitive  tasks  will  be  automated  reducing  the  amount  of  time  necessary  to 
prepare  reports,  write  memos,  gather  data,  etc. 

•  The  software  maze  will  be  eliminated 

All  actions  will  be  do-able  from  KWS.  This  reduces  the  time  and  complexity  of 
moving  from  software  environment  to  software  environment. 

•  Access  to  other  equipment  will  be  simplified 

The  KWS  environment  will  reduce  the  time  and  effort  involved  in  accessing 
equipment,  data  and  facilities. 


An  IE  Toolset  for  KWS 

Applying  IE  methodology  to  the  implementation  of  KWS  is  not  a  simple  process  — you 
can’t  simply  pick  one  from  column  A,  one  from  column  B,  and  so  on.  However,  it 
doesn’t  need  to  be  a  difficult  process  either.  The  use  of  a  “Toolset”  is  what  reduces  the 
difficulty  of  the  process.  We  understand  the  workflow  by  using  a  structured 
interviewing  process.  A  layered  survey  gains  the  value  information  we  need  and  a 
spreadsheet  calculates  the  productivity  value  we  are  looking  for.  The  first  tool,  the 
structured  interviewing  process,  has  already  been  referenced.  The  second  and  third 
tools  will  be  discussed  below. 

We  must  elaborate  on  the  values  that  the  organization  has  and  apply  them  to  the  KWS 
application  benefits.  We  would  conduct  a  survey,  or  rather  a  layered  survey,  where  we 
first  seek  to  establish  a  consensus  on  the  general  vision  of  the  organization.  From 
there  we  then  proceed  to  the  next  layer  and  establish  consensus  on  the  weights  the 
organization  would  apply  to  the  various  value  classes.  The  third  layer  would  entail  a 
detailing  of  the  value  classes  and  weighing  each  of  them.  From  this  information,  we 
would  be  able  to  transfer  the  weights  to  a  spreadsheet  that  would  be  linked  to  the 
KWS  benefits.  A  fourth  layer  would  be  to  estimate  the  potential  for  each  KWS  benefit. 
Some  benefits  depend  for  their  relevance  on  the  organization’s  environment.  For 
example,  if  the  reduction  in  the  access  to  external  references  is  one  of  the  KWS 
benefits,  but  the  subject  organization  never  does  this,  there  is  no  potential  to  this 
benefit  in  this  environment.  Once  this  information  has  been  established  and  entered 
to  the  spreadsheet,  it  would  automatically  produce  a  score  for  the  implementation  of 
KWS  in  the  organization.  Important  to  this  process  is  getting  an  estimate  from  the 
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organization  of  how  well  it  feels  it  currently  performs  or  satisfies  each  of  the  values. 
This  would  produce  the  current  rating  to  compare  with  the  KWS  rating. 


Summary 

This  approach  to  evaluating  the  benefits  of  KWS  in  a  particular  environment  has  a 
very  broad  range  of  application.  It  can  be  done  very  simply  in  a  very  short  time  and 
yet  produce  a  good  evaluation.  It  can  also  be  done  in  a  more  complete  manner  to 
produce  more  detailed  and  better  evaluations.  As  such,  the  amount  of  effort  invested 
in  the  evaluation  need  not  determine  the  applicability  of  the  results.  The  quality  of 
the  effort  is  much  more  important.  This  emphasis  is  quite  different  from  many 
evaluation  techniques  where  the  quantity  of  effort  is  the  controlling  factor. 

The  participation  of  the  members  of  the  area  being  studied  is  crucial  with  the  IE 
approach.  Taken  farther  these  individuals  could,  with  some  assistance,  conduct  the 
studies  themselves.  Reaching  concensus  among  study  participants  is  a  significant  by¬ 
product  of  the  IE  process. 
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Appendix  C:  Time  Logging  Introduction  and 
Instructions  for  Knowledge  Workers 


Introduction 

A  select  group  of  HQUSACE  managers  and  professionals  has  been  cooperating  with 
USACERL  in  the  initial  development  of  the  Knowledge  Worker  System,  a  PC-  and 
LAN-based  computer  application  for  supporting  and  automating  some  of  their  admini¬ 
strative  tasks.  The  members  of  the  initial  KWS  user  group  are  key  players  in  the 
PPBES  cycle. 

Because  the  development,  implementation  and  possible  future  extensions  of  KWS 
absorb  scarce  Army  resources,  it  is  important  that  KWS  be  cost  justified.  This  involves 
identifying  and  measuring  the  various  benefits  and  costs  associated  with  the  system. 
The  best  approach  to  doing  this  is  to  begin  with  a  baseline  work  analysis  (to  document 
work  patterns  prior  to  implementing  KWS),  and  then  to  analyze  work  patterns  after 
KWS  has  been  implemented.  A  comparison  of  these  two  work  pattern  snapshots  will 
serve  as  the  basis  for  identifying  and  quantifying  the  impact  of  the  system. 

The  economic  benefits  of  knowledge  work  frequently  are  difficult  to  quantify  because 
most  knowledge  work  is  complex  in  content  and  impact.  However,  a  new  approach  for 
measuring  the  impact  of  productivity  interventions  (such  as  KWS)  on  knowledge  work 
has  been  developed  and  used  successfully  in  several  dozen  recent  private  sector 
studies.  This  new  methodology  is  called  Work  Profile  Analysis.  It  is  based  on  the 
simple  but  powerful  concept  of  the  “intellectual  content”  of  work. 

In  general,  organizations  pay  their  employees  based  on  the  intellectual  content  of  the 
work  that  they  are  capable  of  performing.  Engineers  with  advanced  training  and 
much  experience  are  paid  more  than  new  and  inexperienced  engineers.  An  engineer 
with  management  training,  ability  and  experience  is  paid  more  than  an  engineer 
without  those  qualifications.  Similarly,  program  and  budget  analysts  with  more 
training  and  experience  are  paid  more  than  those  with  less.  And  of  course,  analysts 
and  engineers  are  paid  more  than  secretaries  and  clerks. 

At  the  same  time,  workers  do  not  spend  100  percent  of  their  work  time  doing  the  work 
that  their  training  and  experience  qualifies  them  to  do.  For  example,  managers, 
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engineers  and  analysts  may  spend  only  a  fraction  of  their  time  in  work  that  could,  in 
principle,  be  delegated  to  lesser  skilled  and  lesser  paid  employees.  Indeed,  research 
has  shown  that  pattern  to  be  the  norm  rather  than  the  exception.  Of  course,  in  many 
organizations,  it  is  the  shortage  of  workers  to  whom  work  may  be  delegated  that  is 
responsible  for  this  pattern  of  work.  For  example,  and  quite  simply,  there  often  are 
not  enough  secretaries  and  clerks  in  an  office  to  handle  all  the  secretarial  and  clerical 
tasks,  and  therefore  many  of  those  tasks  must  be  done  by  professionals  and  managers. 

By  analyzing  how  workers  at  each  position  in  an  organizational  hierarchy  typically 
spend  their  time  (in  terms  of  the  intellectual  content  of  the  work  that  they  are  doing), 
and  by  factoring  into  the  analysis  the  typical  number  of  hours  worked  and  the  total 
cost  (wages  or  salary,  overtime,  fringe  benefits)  of  those  workers,  the  actual  (implicit) 
cost  of  different  types  of  work  (management  work,  engineering  work,  analysis  work, 
secretarial  work, ...)  in  that  organization  can  be  calculated. 

As  a  simple  example  of  this  methodology,  suppose  workers  in  an  engineering 
organization  typically  spend  their  time  as  shown  in  Table  Cl.  For  simplicity,  assume 
that  everyone  works  40  hours  per  week  for  46  weeks  per  year  (this  assumes  6  weeks 
total  of  vacation,  holidays,  sick  days  and  training  days  per  employee).  Finally,  suppose 
that  the  average  annual  total  cost  (salary  or  wages  plus  fringe  benefits)  to  the 
organization  of  a  person  in  each  position  is  given  in  the  last  column  of  the  table. 


Example  Work  Profile  Matrix 

With  these  assumptions,  the  implicit  cost  to  the  organization  of  management  level 
work  is  $76.49  per  hour,  of  professional  engineering  is  $51.33  per  hour,  and  of 
administrative  support  work  is  $18.12  per  hour.  (Determining  these  values  involves 
a  mathematical  economic  model.  You  can  check  the  figures  by  noting  that  the  values 
uniquely  and  exactly  account  for  the  average  salary  in  each  position.)  To  check  these 
values  for  managers,  note  that  managers  spend  552  hours  per  year  doing  management 
level  work  (46  weeks  x  40  hours/week  x  30%),  736  hours  per  year  doing  engineering 
work  (46  weeks  x  40  hours/week  x  40%),  276  hours  per  year  doing  support  work 


Table  Cl.  Sample  work  profile  matrix. 


Management 

Work 

Professional 
Engineering  Work 

Administrative 
Support  Work 

Nonproductive 

Work 

Average 
Annual  Cost 

Managers 

30% 

40% 

15% 

15% 

$85,000 

Engineers 

0% 

60% 

25% 

15% 

$65,000 

Secretaries 

0% 

0% 

90% 

10% 

$30,000 
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(46  weeks  x  40  hours/week  x  15%),  and  276  hours  per  year  in  nonproductive  work 
(46  weeks  x  40  hours/week  x  15%).  Then  note  that  :  (552  hours  of  management  x 
$76. 49/hour)  +  (736  hours  of  engineering  x  $51. 33/hour)  +  (276  hours  of  support  work 
x  $18.12/hour)  +  (276  hours  of  nonproductive  time  x  $0.00/hour)  =  $85,000.  Similar 
checks  can  be  made  for  the  engineers  and  secretaries  in  our  example. 

These  implicit  costs  can  be  used  to  estimate  the  value  of  a  computer  system,  such  as 
KWS.  Continuing  the  simple  illustrative  example,  suppose  that  KWS  could  absorb  60 
percent  of  the  administrative  support  work  and  33.3  percent  of  the  nonproductive  work 
done  by  engineers,  and  suppose  that  the  saved  time  is  redirected  into  professional 
engineering  work.  The  new  row  in  the  work  profile  matrix  would  be: 

Engineers  0%  80%  10%  10%  $65,000 

The  value  to  the  organization  of  this  shift  in  the  engineers’  work  profile  would  be 
$18,889.44  per  engineer  per  year  (20%  x  40  x  46  =  368  additional  hours  of  engineering 
work  per  engineer  per  year  @  $51.33  per  hour.)  This  calculation  assumes  that  the 
additional  engineering  time  is  productive.  That  is,  it  assumes  that  there  is  a  con¬ 
tinuing  “backlog”  of  engineering  work  to  do,  and  that  this  additional  engineering  work 
is  as  worthwhile  as  the  other  engineering  work. 

In  reality,  of  course,  actual  cases  are  far  more  complex  than  the  example  described 
above.  Nonetheless,  these  same  ideas,  incorporated  in  more  complex  models,  can  be 
(and  have  been)  successfully  used  to  evaluate  virtually  any  productivity  intervention 
in  a  knowledge  work  environment. 


This  Study 

The  information  needed  for  this  study  is  “time  log”  data,  that  is,  data  on  how  everyone 
spends  their  time.  We  will  use  this  data  to  develop  a  productivity  snapshot  of  several 
USACE  Headquarters  offices.  The  data  collection  and  analysis  is  based  on  two  critical 
assumptions: 

1.  That  individuals  are  experts  in  their  own  areas  of  work,  that  they  are  currently 
doing  their  jobs  as  well  as  possible,  but  that  their  performance  may  be  limited  or 
impeded  by  factors  such  as  the  type  and  amount  of  technological  support  that 
they  have  available,  by  various  established  policies  and  procedures,  by  the  per¬ 
formance  of  other  government  organizations,  or  by  an  incorrect  level  of  staffing. 
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2.  That  the  data  that  we  collect  on  any  one  individual  is  not  necessarily  representa¬ 
tive  of  how  that  individual  typically  spends  his  or  her  time.  Only  by  combining 
the  data  for  all  persons  in  a  certain  position  (e.g.,  engineer  or  analyst)  will  a 
statistically  significant  work  profile  emerge. 

To  complete  the  study,  we  need  your  cooperation  in  completing  time  logs.  We  ask  that 
you  make  your  entries  on  your  time  log  sheet  once  each  hour  for  your  assigned  three 
consecutive  work  days. 

So  that  we  get  an  accurate  picture  of  the  work  in  your  organization,  it  is  important 
that  you  conscientiously  complete  the  time  logs.  After  reading  these  instructions  and 
completing  your  first  entries  will  take  only  a  minute  or  two. 


Categories  of  Work  in  the  USACE  Headquarters 

There  are  six  major  broad  types  of  work  in  the  organization,  along  with  several 
important  finer  distinctions.  The  six  major  work  categories  are: 

1.  Managerial  and  supervisory  tasks 

2.  Professional  engineering  tasks 

3.  Program  and  budget  analysis  tasks 

4.  Technical  support  tasks 

5.  Administrative  tasks 

6.  Tasks  related  to  work  on  government  committees. 

Each  of  these  categories  of  tasks  is  explained  in  detail  in  “Appendix  D:  Dictionary  of 
Work  Categories  and  Example  Tasks.”  In  addition  to  these  six  major  categories  of 
tasks,  there  are  several  other  activities  which  normally  occupy  the  time  of  HQUSACE 
personnel.  These  are:  professional  development,  nonproductive  time,  community 
service,  travel,  and  personal  time. 

Finally,  for  these  few  days,  completing  the  time  log  is  an  activity  which  will  take  a  few 
minutes  of  each  hour.  These  activities  are  explained  in  the  same  document.  Tasks 
usually  are  composed  of  distinct  sub-tasks.  For  example,  the  task  of  preparing  an 
engineering  analysis  might  include  the  following  sub-tasks: 

1.  Identify  and  access  the  appropriate  procedures 

2.  Read  and  understand  the  procedures 

3.  Gather  the  necessary  data  and  information 

4.  Review  the  data  and  information  for  completeness 
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5.  Locate  and  access  additional  information  if  needed 

6.  Perform  the  required  engineering  analysis 

7.  Prepare  a  draft  report  of  findings  and  conclusions 

8.  Discuss  analysis,  findings,  and  conclusions  with  supervisor 

9.  Prepare  the  final  report,  including  tables  and  graphics 

10.  Make  copies  of  the  report  for  distribution 

11.  Forward  the  copies  to  the  appropriate  offices 

This  task  would  be  considered  an  engineering  task  because  one  or  more  of  the  sub¬ 
tasks  must  be  performed  by  an  engineer.  However,  some  of  the  sub-tasks  could  be 
characterized  as  “support”  or  “set-up”  work;  and  some  of  the  sub-tasks  could  be 
characterized  as  “principal”  or  “core”  work.  For  example,  sub-tasks  a,  c,  d,  e,  i,  j,  and 
k  are  partly  or  completely  “support”  level  work.  That  is,  these  are  sub-tasks  that  could 
be  delegated  to  a  competent  and  properly  trained  administrative  assistant,  if  one  were 
available.  On  the  other  hand,  items  b,  f,  g,  and  h  are  “principal”  level  sub-tasks.  They 
involve  professional  engineering,  and  they  cannot  be  delegated  to  a  support  worker. 

In  general,  the  “support”  or  “set-up”  sub-tasks  of  a  professional  or  managerial  task 
involve  activities  like  searching,  identifying,  accessing,  looking  up,  downloading, 
locating,  transmitting  or  communicating  documents,  data  or  information;  or  they 
involve  routine  office  skills  like  typing,  filing,  data  entry,  simple  database  queries, 
scheduling,  copying,  faxing,  etc. 

As  you  complete  your  time  log,  you  will  be  asked  to  distinguish  between  time  spent  in 
“principal”  level  work  and  time  spent  in  “support”  level  work. 

Engineering  and  Program/Budget  Analysis  work  also  can  be  classified  according  to  the 
minimum  level  of  education,  training  and  experience  necessary  to  accomplish  that 
work.  In  the  HQUSACE,  we  can  distinguish  two  levels  of  engineering  and  two  levels 
of  program/budget  analysis  work.  These  are: 

•  Junior  Engineer/Analyst  Level  Work: 

This  is  work  that  can  be  done  by  a  competent  entry  level  engineer  or  analyst. 
This  is  someone  with  the  educational  background  appropriate  to  the  profession, 
who  has  had  the  appropriate  training  but  who  has  less  than  several  years 
experience.  Junior  engineering  or  analyst  level  work  would  include  much  of  the 
routine  engineering  or  analysis  work  performed  in  the  HQUSACE.  As  a  general 
guide,  junior  level  engineering  work  would  be  professional  engineering  work  that 
could  be  delegated  to  a  GS-12  or  lower  engineer.  Junior  level  program/budget 
analysis  work  would  be  analysis  work  that  could  be  delegated  to  a  GS  11  or  lower 
analyst. 
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Senior  Engineer/Analyst  Level  Work: 

This  is  either  engineering  or  analysis  work  of  sufficient  complexity,  uniqueness, 
impact  or  importance  that  it  should  be  handled  by  engineers  or  analysts  with 
more  than  several  years  experience;  or  it  is  planning,  directing  and  overseeing 
the  engineering/analysis  work  of  others.  As  a  general  guide,  senior  level  engi¬ 
neering  work  would  be  professional  engineering  work  that  should  be  performed 
by  GS  13  or  higher  engineers.  Senior  level  program/budget  analysis  work  would 
be  analysis  work  that  should  be  done  by  GS  12  or  higher  analysts. 


Instructions 

Please  look  at  a  time  log  sheet.  The  first  column  on  the  left  lists  the  work  categories 
discussed  above.  Listed  first  are  the  six  major  types  of  work  (with  engineering  and 
program/budget  analysis  each  separated  into  junior  and  senior  level  work),  then  the 
remaining  activities  which  are  sometimes  a  significant  part  of  nearly  everyone’s  day, 
then  a  catch-all  “Other”  category,  and  finally  a  row  labeled  TOTAL  MINUTES. 

The  rest  of  the  columns  on  the  sheet  are  for  your  entries  for  each  daily  time  period. 
The  first  and  last  of  the  columns  on  the  sheet  are  for  your  entries  for  each  daily  time 
period.  The  first  and  last  of  these  time  periods  are  open  ended  (BEFORE  7:00  AM  and 
AFTER  6:00  PM),  and  the  others  are  exactly  1  hour  long. 

Before  using  the  time  log  sheet  to  record  your  daily  activities,  you  should  review  the 
“HQUSACE  Dictionary  of  Work  Categories  And  Example  Tasks”  (Appendix  D).  This 
will  help  you  understand  how  each  category  is  being  defined  for  the  purpose  of  this 
study.  It  will  also  ensure  consistency  in  everyone’s  interpretation  of  the  work 
categories. 

Make  your  entries  in  the  time  log  sheet  at  the  end  of  each  hour  (or  as  close  to  that  time 
as  possible).  Enter  the  number  of  minutes  of  each  type  of  work  for  that  time  period  in 
the  appropriate  space  on  the  time  log  sheet. 

The  work  categories  listed  in  the  time  log  sheet  are  intended  to  be  complete  and 
nonoverlapping.  Therefore,  the  entries  in  each  column  should  add  up  to  60  minutes. 
The  only  exception  might  be  your  first  and  last  time  periods  of  each  day.  For  example, 
if  you  start  work  at  8:40  AM  or  leave  work  at  4:30  PM,  then  your  first  and  last  periods 
will  contain  only  20  minutes  and  30  minutes,  respectively.  Please  be  sure  to  check 
your  entries  so  that  you  have  accounted  for  all  your  time  in  each  column.  In 
particular,  you  should  account  for  all  60  minutes  in  each  hour  that  is  not  the  first  or 
last  hour  of  the  work  day. 
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In  most  cases,  assigning  your  actual  work  in  the  time  log  categories  should  be 
straightforward.  However,  because  work  in  HQUSACE  is  varied  and  complex,  not 
every  work  activity  can  be  explicitly  anticipated  by  the  time  log.  Therefore,  in  those 
cases,  you  should  first  review  the  document,  HQUSACE  DICTIONARY  OF  WORK 
CATEGORIES  AND  EXAMPLE  TASKS,  and  then  use  your  best  judgement  to  assign 
your  time  to  the  closest  category.  Note  that  the  listed  tasks  under  each  main  type  of 
work  are  intended  only  as  examples.  If  the  work  that  you  did  reasonably  fits  that 
category,  you  should  log  your  time  under  that  category  even  if  your  specific  tasks  are 
not  explicitly  listed.  However,  if  none  of  the  categories  truly  fit  the  work  that  you  did, 
then  use  the  “OTHER”  category  and  add  a  brief  explanation  on  the  back  of  the  sheet. 

As  a  guide  in  assigning  your  time  to  the  proper  category,  use  the  delegation  test.  Ask 
yourself:  What  is  the  lowest  level  to  which  this  work  could  reasonably  be  delegated? 
If  the  work  could  be  done  by  a  secretary  or  clerk,  then  that  work  is  administrative 
support  work.  If  the  work  could  not  be  done  by  a  typical  secretary  or  clerk,  but  it  could 
be  done  by  someone  with  some  technical  training  (e.g.,  a  PC  specialist),  then  the  work 
is  technical  support  work.  If  the  work  is  engineering  work  but  it  could  be  delegated 
to  a  junior  engineer,  then  the  work  is  junior  level  engineering  work.  And  so  on. 

In  posing  and  answering  this  delegation  question,  don’t  worry  about  current  staffing 
limitations  (that  is,  for  example,  a  secretary  could  do  this  s  but  none  are  available). 
Simply  assume  that  there  is  adequate  staffing  to  handle  any  delegated  work. 

If  you  need  help,  or  if  you  have  any  questions  about  filling  out  your  time  log  sheet,  you 
should  call  Beverly  Thomas  at  217/373-7284. 

Please  remember  to  make  your  entries  on  the  form  at  the  end  of  each  hour,  or  as  close 
to  that  time  as  possible! 


Critical  Information 

Please  do  not  make  up  data  if  you  forget  to  log  a  certain  period.  Rather,  we  will  extend 
your  assigned  3-day  logging  interval  so  that  a  total  of  3  days  work  is  ultimately 
captured. 

Please  do  not  rely  on  your  memory  to  fill  out  the  form  only  once  or  twice  a  day,  instead 
of  once  each  hour.  Extensive  experience  clearly  shows  that  hourly  logs,  while  an 
admitted  inconvenience,  are  the  best  way  to  produce  accurate  results. 
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Do  not  be  concerned  if  the  days  which  you  log  do  not  appear  to  be  “typical”  for  you.  Do 
not  adjust  your  data  to  make  it  look  typical!  We  fully  expect  that  no  one’s  3  days  will 
be  typical  for  him  or  her.  However,  by  combining  the  results  from  all  employees,  the 
computed  averages  will  be  statistically  reliable. 

Do  not  be  reluctant  to  log  nonproductive  time.  This  category  helps  us  understand  and 
quantify  your  need  for  additional  support.  Remember,  our  assumption  is  that  you  are 
getting  your  job  done  the  best  way  possible,  even  if  that  entails  spending  a  lot  of  time 
on  nonproductive  activities. 

Your  time  log  sheets  will  be  treated  confidentially.  The  time  log  sheets  will  be  analyzed 
by  an  external  consultant.  Only  summarized  data  (no  names)  will  be  reported. 
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Appendix  D:  Dictionary  of  Work  Categories 
and  Example  Tasks 


Project  Administration 

These  are  project-related  tasks  (research  or  reimbursable)  that  must  be  performed  by 
the  project  manager.  These  can  be  referred  to  as  “plan,  program,  schedule  and  budget” 
project.  Some  examples  of  this  work  include: 

•  Preparing  PSBs  or  RDMIS 

•  Reconcile  detailed  cost  listing 

•  Developing  proposals  for  reimbursable  work 

•  Developing  CPAR,  SBIR,  QRIP  programs 

•  Developing/planning  USACERL-wide  programs 

•  Providing  technical  advice  to  other  government  agencies 

•  Responding  to  ad  hoc  technical  inquiries 

•  Developing  material  for/briefing  to  technical  sponsors  and/or  DRD 

•  Developing  (or  assisting  TL  in  developing)  MADs  or  LRST  strategies 

•  Responding  to  ad  hoc  Plans  and  Programs  requests. 


Project  Execution 

These  are  tasks  that  directly  support  a  research  or  reimbursable  project.  These  can 
be  referred  to  as  “execute”  research  or  reimbursable  project.  Some  examples  of  this 
work  include: 

•  Evaluating  the  results  of  other  research  for  applicability  to  your  project 

•  Developing  technical  specifications  and  statements  of  work  for  contracts 

•  Preparing/refining  detailed  project  plan 

•  Defending  and  justifying  plans  to  senior  management  or  colleagues 

•  Designing/refining  models 

•  Designing/developing  prototypes 
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Computer  programming,  debugging,  testing 
Reviewing/correcting  subordinate’s  project-related  work 
Assigning  project  responsibilities  to  subordinates 
Hiring/training  personnel  to  work  on  project. 


Technology  Transfer 

These  are  tasks  that  lead  to  the  dissemination  of  information  about  research  or 
reimbursable  project.  Some  examples  of  this  work  include: 

•  Presenting  research  findings  to  users/customers/other  researchers 

•  Participating  in  project-related  conferences 

•  Providing  training  to  users 

•  Writing  technical  reports,  PR  materials,  or  technical  articles 

•  Writing  technical  and/or  user  documentation 

•  Preparing/revising  technology  transfer  plan  documents. 


Technical  Support  Tasks 

These  are  support  tasks  which  demand  some  technical  skill  or  knowledge.  Some  of 
these  tasks  are  done  by  nearly  everyone.  The  key  question  to  ask  is,  “Is  this  work 
related  to  my  project?”  These  tasks  should  not  be  directly  related  to  your  reimbursable 
or  research  project. 

Some  examples  include: 

•  Computer  problem  solving 

•  Installing/fixing/configuring  computer  hardware  or  software 

•  Doing  routine  maintenance  on  computer  hardware 

•  Performing  database  maintenance 

•  Providing  formal  or  informal  computer  training  for  others 

•  Using  a  PC  to  produce  reports,  or  to  query  or  update  a  database 

•  Performing  statistical  analyses 

•  Producing  technical  charts  or  graphs. 
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Administrative  Support  Tasks 

These  are  nontechnical  tasks  that  can  be  performed  by  someone  with  general  office 
skills  and  some  on-the-job  training.  Some  of  this  work  is  done  by  nearly  everyone. 
Some  examples  include: 

•  Opening,  reading,  routing  mail  and  correspondence 

•  Routing  requests  and  information  to  others 

•  Responding  to  requests  for  routine  information 

•  Preparing  time  sheets 

•  Preparing  status  reports,  e.g.,  Weekly  Accomplishments 

•  Data  entry,  e.g.,  Inventory 

•  Answering  phone,  routing  calls,  taking  messages 

•  Reading  company  memos,  policies,  new,  etc. 

•  Filing  documents,  forms,  etc 

•  Faxing 

•  Scheduling  meetings 

•  Making  travel  arrangements 

•  Copying,  collating  and  distributing  materials 

•  Ordering  hardware/software/supplies  via  credit  card,  etc. 

•  Doing  vacation/sick  leave  paperwork 

•  Preparing  travel  vouchers 

•  Cleaning  office  or  team  area 

•  Preparing  simple  graphs  and  charts 

•  Escorting  visitors. 


Serving  on  or  Supporting  Government  Committees 

This  refers  to  all  the  time  that  you  spend  working  on  government  committees  that  are 
not  project-related.  This  work  might  be  done  alone  or  in  meetings  with  others.  Some 
examples  of  these  tasks  include: 

•  Developing  agendas 

•  Participating  in  TQM  activities,  e.g.,  PATs  or  Focus  Groups 

•  Reviewing  minutes 

•  Setting  schedules 

•  Attending  meetings 

•  Gathering  or  reviewing  information 

•  Preparing  reports  or  presentations 

•  Follow-up  phone  calls,  memos,  etc. 
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Professional  Development 

This  refers  to  all  the  time  you  spend  gaining,  maintaining  or  updating  the  knowledge 
and  skills  necessary  to  do  your  job.  This  includes  all  formal  and  informal  training  and 
education.  This  work  might  be  done  alone  or  in  group  sessions.  Some  of  this  work  is 
done  by  nearly  everyone.  Some  examples  include: 

•  Reading  professional  or  trade  literature 

•  Learning  or  experimenting  with  computer  software  and  hardware 

•  Attending  seminars,  conferences,  workshops  or  training  classes 

•  Receiving  informal  training 

•  Participating  in  professional  organizations/committees. 


Nonproductive  Time 

This  refers  to  work  time  during  which  no  work  gets  done.  Despite  everyone’s  best 
efforts,  nonproductive  time  is  an  almost  inevitable  part  of  each  day.  Some  typical 
nonproductive  activities  include: 

•  Walking  to  a  meeting  and  waiting  for  it  to  start 

•  Searching  through  files  for  a  lost  document 

•  Searching  for  someone  or  something 

•  Waiting  in  line  to  use  a  copy  or  Fax  machine 

•  Waiting  to  see  someone 

•  Telephone  tag 

•  Fire  drills. 


Community  Service  Activities 

This  refers  to  all  the  time  that  you  spend  during  working  hours  engaged  in  community 
support  activities  as  a  representative  of  the  government.  This  work  might  be  done 
alone  or  in  meetings  with  others.  Some  examples  of  these  tasks  include: 

•  Fund  raising  calls,  e.g.,  CFC 

•  Attending  public  meetings 

•  Administrative  work  on  behalf  of  a  community  organization. 
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Travel  Time 

This  refers  to  time  that  you  spend  traveling  on  company  business  during  normal 
working  hours,  during  which  time  you  are  not  getting  any  work  done.  If,  for  example, 
you  are  getting  work  done  while  you  are  riding  in  a  car  or  airplane,  then  you  should 
log  that  time  under  the  appropriate  work  category.  Do  not  include  intra-building 
transit  time  in  this  category  (log  that  time  in  the  nonproductive  category),  but  do 
include  inter-building  transit  time  (e.g.,  going  from  Building  #1  to  Building  #3). 


Personal  Time 

This  refers  to  the  nonwork  periods  during  the  work  day.  Examples  include: 

•  Lunch  time 

•  Coffee  breaks 

•  Networking/personal  discussions 

•  Restroom  breaks 

•  Personal  phone  calls 

•  Personal  errands. 


Completing  the  Time  Log 

This  refers  to  the  time  you  spend  reading  these  instructions  and  filling  out  time  log 
sheets.  If  you  spend  time  referring  back  to  these  instructions  or  documents,  getting 
help  in  completing  the  time  log,  distributing  or  collecting  these  forms  or  reviewing 
them  for  completeness,  or  helping  someone  else  with  their  form,  log  that  time  under 
Completing  Time  Log,  as  well. 


Meetings 

Use  this  area  to  indicate  any  meetings  that  do  not  fall  under  other  categories.  For 
example,  SAEDA  security  briefings,  Division  meetings,  information  meetings  about 
some  work-related  (but  nonproject)  related  topic  such  as  the  new  USACERL  phone 
system,  etc.  Many  of  your  meetings  that  are  not  project  related  will  be  logged  either 
here  or  under  “Professional  Development.” 
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Other 

Please  note — on  a  piece  of  paper  or  separate  file — any  activities  that  you  have 
performed  during  the  past  hour  that  do  not  fit  under  any  other  category  described 
above.  Record  the  #  of  minutes  spent  in  this  undefined  category  in  the  “Other  Type  of 
Work”  section. 
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